Gitlab pipeline for End to End and parametric scenarios (WIP)#6651
Draft
nccatoni wants to merge 233 commits into
Draft
Gitlab pipeline for End to End and parametric scenarios (WIP)#6651nccatoni wants to merge 233 commits into
nccatoni wants to merge 233 commits into
Conversation
Contributor
|
|
nccatoni
force-pushed
the
nccatoni/gitlab
branch
6 times, most recently
from
17e7dfc to
047676e
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
from
047676e to
06cd30f
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
from
448583b to
81f74d6
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
from
0cecfb2 to
314ca4a
Compare
|
nccatoni
force-pushed
the
nccatoni/gitlab
branch
from
e4805ca to
624134d
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
6 times, most recently
from
c7fe9a2 to
ad45877
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
2 times, most recently
from
9384467 to
ed748cf
Compare
nccatoni
force-pushed
the
nccatoni/gitlab
branch
from
ed748cf to
17aca81
Compare
The trigger jobs (.run_test_pipeline_base) now need the binaries_artifact job (optional) so the child pipeline doesn't start before the artifact is ready. Without this the child pipeline fired immediately and the cross-pipeline needs failed to retrieve artifacts. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Job names containing ':' and '[' (e.g. parallel matrix instance names like 'build linux: [amd64, cp311-cp311, ...]') are special YAML characters and must be quoted, otherwise the generated pipeline YAML is invalid. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
build_test_pipeline needs the binaries_artifact job (optional, so it's a no-op when the input is empty). Since run_test_pipeline_* always waits for build_test_pipeline before triggering the child pipeline, this guarantees the artifact is ready before any child job tries to download it via pipeline: \$PARENT_PIPELINE_ID. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
build_test_pipeline only needs to wait for the job to finish before generating the child pipeline; it never uses the artifacts itself. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
\$CI_PARENT_PIPELINE_ID is provided automatically by GitLab in any triggered child pipeline. The manually-passed PARENT_PIPELINE_ID was redundant and potentially empty if not resolved correctly, causing the artifact download to fail. Since needs:pipeline: waits for the referenced job itself, build_test_pipeline also doesn't need to wait for binaries_artifact. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
needs:pipeline: in child jobs does NOT wait — it just tries to download the artifact and fails if the upstream job is not done yet. build_test_pipeline must therefore wait for the binaries_artifact job (optional, no-op when the input is empty) so the artifact is guaranteed to exist before any child pipeline job runs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The generated child pipeline jobs use pipeline: \$UPSTREAM_PIPELINE_ID to reference the parent pipeline's binaries artifact. This variable was never being set, so the artifact download had an empty pipeline reference and failed. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Different weblogs need different python version wheels. Add binaries_artifacts (semicolon-separated, since job names contain commas) as the full list of upstream artifact jobs to download from in the generated child pipeline. The existing single-job binaries_artifact remains the timing gate. Falls back to [binaries_artifact] when binaries_artifacts is empty, so existing callers that only set binaries_artifact are unaffected. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Changes
Potential issue
Rebuilding of CI images is susceptible to race conditions if two PRs modifying different dependencies of the image are merged to main without being rebased on one another before. The right way to solve this issue would be to implement a merge queue. This is still unlikely in the current configuration and should be caught fairly fast by running the workflow on main.
Workflow
🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist
tests/ormanifests/is modified ? I have the approval from R&P teambuild-XXX-imagelabel is present