Repository navigation
Add a workflow, run by hand, that measures .test_durations on a CI runner - #843
Merged
Merged
Conversation
…nner It runs the whole coverage check unsharded with --store-durations --clean-durations, set up as the coverage job is, and uploads .test_durations to commit. Local timings differ from a CI runner's in speed, Python version, and optional dependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
CI splits the per-module coverage check into shards, balanced by the per-test-file durations in
.test_durations(seechecks/coverage_.py --shard). These durations go stale as tests change. They have changed most recently because the coverage check now runs qLDPC's compiled functions as plain Python (#839). Measuring them on a developer's machine is unreliable, since it can differ from a CI runner in speed, Python version, and optional dependencies. For example, thegapextra installs only on Python 3.13, while CI runs 3.14.Changes
.github/workflows/test-durations.yamlis a workflow that runs only when started from the Actions tab. It sets up the environment as the coverage job does, runs the whole coverage check unsharded with--store-durations --clean-durations, and uploads.test_durationsas an artifact to commit. The upload setsinclude-hidden-files, since the action skips files whose names start with a dot.continuous-integration.yamlpoints to the new workflow.Additional notes
The workflow copies the coverage job's set-up, so a change to one should be made to the other; its header says so. Once this merges, one run of the workflow produces the new
.test_durations, which a follow-up PR commits.