gh-984: Add config to run on Archer2 by connoraird · Pull Request #1064 · glass-dev/glass

connoraird · 2026-05-11T10:12:46Z

Description

Adds submission scripts and documentation of how to run the regression tests on Archer2.

Testing

I have tested the archer2 script by running a regression test with fd80e8f as the starting version and main as the final version and received an error due to the change in the glass modules which causes benchmarks for fd80e8f to fail.
I have also ran comparing main to connor/run-on-archer2 which runs as expected.

Refs: #984
Closes: #984

Checks

Is your code passing linting?
Is your code passing tests?
Have you added additional tests (if required)?
Have you modified/extended the documentation (if required)?
Have you added a one-liner changelog entry above (if required)?

…single script

paddyroddy

Small comment on the files itself, will now try on ARCHER2

…cript

paddyroddy

Updates after running it more

paddyroddy · 2026-06-02T15:34:20Z

+Now we have cloned glass, we can run the script
+[run_regression_test.sh](./run_regression_test.sh) which will setup the required
+environments and submit regression test job to slurm. A help message is
+provided. Just run `run_regression_test.sh -h` from the root of the GLASS repo.


> run_regression_test.sh -h -bash: run_regression_test.sh: command not found

The instructions aren't clear IMO. It should say something like:

cd glass/tests/benchmarks/archer2 bash run_regression_test.sh

or

bash glass/tests/benchmarks/archer2/run_regression_test.sh

or

cd "${HOME/home/work}"/glass/tests/benchmarks/archer2 ./run_regression_test.sh

Also can we give them a sample command? Like I'm genuinely not sure how to run this myself. We should make "start" and "end" really unambiguous. Also, -d | --glass-dir <glass/dir> Path to the cloned glass directory. Does that mean I should have this shell script outside the repo?

I've made some changes which hopefully make it clearer

This reverts commit 9f36557.

paddyroddy · 2026-06-03T09:30:58Z

+./tests/benchmarks/archer2/run_regression_test.sh \
+   -d "$(pwd)"                                    \
+   -s main                                        \
+   -e feature                                     \
+   -a myaccount


I got this working with ./tests/benchmarks/archer2/run_regression_test.sh -d "$(pwd)" -s main -e connor/run-on-archer2 -a ecsega23.

Maybe we should document that it's ecsega23 rather than proddy?

Also is the -d mandatory here? Is there a way we could detect that we're already in the directory?

Maybe we should document that it's ecsega23 rather than proddy?

I've not been specific about the account as this is only for the current grant code. In the future it will likely change to a different account code.

Also is the -d mandatory here?

It is. When a job is submitted the script is saved and ran from somewhere else which makes determining paths a little more complicated. We could default to pwd but I think that's not very intuitive and would only work if running from the root of the glass dir . However, I could be convinced if you feel strongly.

In the future it will likely change to a different account code.

I agree to not specifying the account. But we should make it clear that "account" is not your username.

However, I could be convinced if you feel strongly.

Agreed that it may be unintuitive. Seems a little odd to me, but there is probably no other way.

Should we document how to get the results? I'm not sure. Could look it up but we might as well document.

Makes sense, I'll have a go

paddyroddy

I've been able to run this now and I think the instructions are complete

connoraird self-assigned this May 11, 2026

connoraird force-pushed the connor/run-on-archer2 branch from 596542c to cad9042 Compare May 11, 2026 16:29

connoraird added 7 commits May 12, 2026 14:28

Add docs and submission script for archer2

ef3166a

Update archer2 submissions script to not use nox

43e07fc

Activate python venv in archer2 submission script

0d6f1c3

Remove batch array config and increase timeout

99d0dc5

Add sub script for archer2 gpu

889aa44

Allow running benchmarks against jax

3cd763f

Setup array backends when running benchmark tests

301871c

connoraird mentioned this pull request May 12, 2026

gh-995: Allow filtering a single benchmark test #1068

Merged

5 tasks

Make test_cls2cov more forgiving

9f3dcf3

connoraird force-pushed the connor/run-on-archer2 branch from c6cd4bd to 9f3dcf3 Compare May 12, 2026 13:53

connoraird added 17 commits May 18, 2026 14:59

Update archer2 workflow to setup environments and submit jobs from a …

3d98788

…single script

Fix typos and paths and use --active in uv sync command

2fa32e5

Improve the interface to the regression test script

f0a844c

Rename script to run_regression_test.sh

f01bf05

Ensure GLASS_DIR is provided

813df5b

Fix setting benchmark dir path in script and rename scripts

8bb77b9

Move account setting to run_regression_test.sh

3f8b74d

Fix typo

53899a8

Remove uv from archer2 scripts

a133cb0

Swap back to uv

f2cb693

Simplify archer2 execution instructions

939f1f2

PWD -> HOME

6e2b9bb

Update work profile file to be correct

7080ca7

Make setting up env optional and improve docs

f32ecab

Remove outptu from run script

c98da36

Ensure active venv is used in submitted jobs

3de8a83

Put --active in the right place

03771a6

connoraird marked this pull request as ready for review May 19, 2026 09:19

Use the correct output name so it is ignored by git

82fb087

This was referenced May 27, 2026

Investigate past regressions #964

Closed

Investigate and fix regression from #874 #967

Closed

paddyroddy reviewed Jun 1, 2026

View reviewed changes

Comment thread tests/benchmarks/archer2/submission_script_cpu.sh Outdated

Comment thread tests/benchmarks/archer2/submission_script_cpu.sh Outdated

paddyroddy reviewed Jun 1, 2026

View reviewed changes

Comment thread tests/benchmarks/archer2/README.md Outdated

Comment thread tests/benchmarks/archer2/README.md Outdated

Comment thread tests/benchmarks/archer2/README.md

connoraird added 2 commits June 2, 2026 14:24

Add correct env path

a315790

Add output file to command rather than sentence and lint submission s…

c2a4e90

…cript

connoraird requested a review from paddyroddy June 2, 2026 13:34

linting

29700bf

paddyroddy reviewed Jun 2, 2026

View reviewed changes

Comment thread tests/benchmarks/archer2/README.md Outdated

paddyroddy requested changes Jun 2, 2026

View reviewed changes

connoraird added 7 commits June 2, 2026 17:23

Remove the need for the user to pass the path to the GLASS dir

9f36557

Correct path

bbcaa1a

typo

f7a3f53

Revert "Remove the need for the user to pass the path to the GLASS dir"

dbced93

This reverts commit 9f36557.

Add example command

5bc9d07

Use pushd and popd to allow running script from any path

b3b63c8

Fix typos and remove excess

a26a9a0

connoraird requested a review from paddyroddy June 2, 2026 16:48

Merge branch 'main' into connor/run-on-archer2

d19f0f0

paddyroddy reviewed Jun 3, 2026

View reviewed changes

Add explanation of account and how to read results

457e065

connoraird requested a review from paddyroddy June 3, 2026 10:07

connoraird and others added 3 commits June 3, 2026 11:16

Add troubleshooting about time limit

084e147

linting

bf3e522

Fix markdown

3cc45b3

paddyroddy approved these changes Jun 3, 2026

View reviewed changes

Fix typo

b7150ac

connoraird merged commit 3dfe047 into main Jun 3, 2026
17 of 19 checks passed

connoraird deleted the connor/run-on-archer2 branch June 3, 2026 11:56

Uh oh!

Conversation

connoraird commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Checks

Uh oh!

paddyroddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paddyroddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paddyroddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

connoraird commented May 11, 2026 •

edited

Loading