Skip to content

Scripts for a compiler matrix#656

Open
artv3 wants to merge 24 commits into
developfrom
artv3/compiler-sweep
Open

Scripts for a compiler matrix#656
artv3 wants to merge 24 commits into
developfrom
artv3/compiler-sweep

Conversation

@artv3

@artv3 artv3 commented Mar 17, 2026

Copy link
Copy Markdown
Member

Summary

  • This PR is adds helper scripts to do cross compiler performance comparisons

Sample run:
See: data/compiler_sweeps/tier1_base_raja_hip_repro.txt

Folks, these scripts are AI generated. But I spent time looking over the code and it looks solid to me. I would like to invite the team to review.

@artv3 artv3 requested a review from a team March 17, 2026 04:58
@@ -0,0 +1,352 @@
#!/usr/bin/env bash

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this script should have a more descriptive name. Not sure what it could be, but worth thinking a bit about.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @rhornung67 , what if we call it the compiler matrix, since that is what we are building?

@artv3

artv3 commented Mar 31, 2026

Copy link
Copy Markdown
Member Author

Consider that we may want different MPI variants

@artv3

artv3 commented Mar 31, 2026

Copy link
Copy Markdown
Member Author

Consider throughput comparisons for different compilers

@artv3

artv3 commented Apr 28, 2026

Copy link
Copy Markdown
Member Author

To compare compiler performance consider putting together a ratio plot between a base compiler

@artv3 artv3 requested review from a team and removed request for a team May 1, 2026 18:47
@artv3 artv3 changed the title add scripts to compare compiler performance for a given list of kernels Draft: add scripts to compare compiler performance for a given list of kernels May 1, 2026
@artv3 artv3 marked this pull request as draft May 1, 2026 18:48
@artv3 artv3 changed the title Draft: add scripts to compare compiler performance for a given list of kernels Scripts for a compiler matrix May 1, 2026
@artv3 artv3 requested review from a team and rhornung67 May 1, 2026 22:46
@artv3 artv3 marked this pull request as ready for review May 1, 2026 22:46
@johnbowen42 johnbowen42 added this to the June 2026 Release milestone May 5, 2026

usage() {
cat <<'EOF'
Usage:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to prefer a shell script over a python script here? I thought maybe it would dovetail better with our burgeoning analysis scripts to do Python

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thoughts are that since there the workflow is environment dependent, it would be more fitting to use a shell script. For example its consistent with the lc-build scripts, handles stuff like srun, cmake, module a bit more naturally

log_dir="${2:-}"
shift 2
;;
-h|--help)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i.e. we could do an argparse here instead of this


echo "==> [${entry_id}] line ${line_no}: $(basename -- "${entry_build_script}") ${script_args[*]}"

# Execute each matrix entry in a fresh login shell so module operations and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this why it's easier to handle this with a shell script?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@rhornung67

Copy link
Copy Markdown
Member

@artv3 would you please add a short section to the RAJAPerf User Guide explaining how to use this?

Comment thread docs/sphinx/user_guide/run.rst Outdated
Comment thread docs/sphinx/user_guide/run.rst Outdated
@MrBurmark

Copy link
Copy Markdown
Member

@artv3 I updated the script to also output a basic html landing page and per kernel html pages in addition to what it was outputting before. This makes it easier to navigate through the output files.
I also change the plots to be log-log.

# Pick a default C++ standard compatible with the CUDA toolchain.
# CUDA 11.x does not support C++20 in NVCC mode (BLT enforces this).
NVCC_MAJOR="${NVCC_COMP_VER%%.*}"
BLT_CXX_STD="c++17"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is relevant anymore.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it is not. We can't use CUDA < 12 when we require C++20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants