[cdac] Share one coreclr build between cDAC dump + stress test legs by max-charlamb · Pull Request #129237 · dotnet/runtime

max-charlamb · 2026-06-10T15:06:12Z

Note

This PR was prepared with assistance from GitHub Copilot CLI.

Summary

Refactors runtime-diagnostics.yml so the cDAC dump tests and stress tests share one coreclr+libs build per platform, and so per-platform pipelines run in parallel (no waiting on slower platforms).

Before

Stage CdacBuild       (8 plats, full build each)
Stage CdacDumpTests   (8 plats, full build each)   <- 8 redundant builds
Stage CdacStressTests (4 plats, full build each)   <- 4 redundant builds
Stage CdacXPlatDumpGen (6 plats, full build each)  <- 6 redundant builds
Stage CdacXPlatDumpTests (6 plats, full build each) <- 6 redundant builds

Each test leg rebuilt coreclr from scratch.

After

Stage Cdac (single stage, jobs in parallel except for stated deps)
  build_<plat>_CdacBuild              x8  (publishes artifacts/bin tar)
  build_<plat>_CdacDumpTest           x8  job-dep CdacBuild_<plat>   [single-leg, !Schedule]
  build_<plat>_CdacStressTest         x4  job-dep CdacBuild_<plat>   [!Schedule]
  build_<plat>_CdacXPlatDumpGen       x6  job-dep CdacBuild_<plat>   [xplat OR Schedule]
Stage CdacXPlatDumpTests              dependsOn: Cdac (intrinsic xplat sync)
  build_<plat>_CdacXPlatDumpTest      x6

One full coreclr+libs build per platform.
Per-platform parallelism: linux_x64 dump tests start as soon as linux_x64 build is done; no waiting on osx_arm64 to finish building.
Pattern follows the canonical runtime.yml dependsOnGlobalBuilds mechanism (see runtime.yml:1591,1622,1653).

How the Release / Debug cDAC split works

The user requested:

Build coreclr in Checked (-rc checked).
Stress tests use the Release native cDAC shim (loaded in-process by the runtime).
Dump tests use the Debug managed cDAC assemblies (referenced directly by the DumpTests project).

These coexist because they live in different artifact locations:

CdacBuild runs -c Release -rc checked → native shim ships in the Release testhost.
Stress test legs keep cdacTestConfig: $(_BuildConfig) (Release) — they pick up the Release native shim from the testhost.
Dump test legs pass new cdacTestConfig: Debug parameter — payload-prep MSBuild walks the DumpTests project graph and rebuilds the managed cDAC assemblies at Debug in a separate output directory.

Files changed

eng/pipelines/runtime-diagnostics.yml — collapsed 5 stages into 2, wired job-level deps.
eng/pipelines/cdac/prepare-cdac-helix-steps.yml — added cdacTestConfig parameter (defaults to $(_BuildConfig) for back-compat).
(Existing Build / SOS stage untouched — different subset, different platforms.)

Validation

Local YAML syntax check passes.
Needs a manual pipeline trigger to validate end-to-end behavior (shared artifact download, per-platform parallelism, Debug-cDAC payload prep on the dump side).

Adds a per-platform CdacBuild stage that builds coreclr (Checked) + libs + tools.cdac + the cDAC dump+stress test subsets, then publishes artifacts/bin as a CdacBuildArtifacts_<plat> tar. Subsequent commits will switch CdacDumpTests, CdacStressTests, and the CdacXPlat stages to consume this artifact instead of each running their own full build. The matrix uses cdacDumpPlatforms (the broader 8-platform set); CdacStressTests' 4-platform subset is still satisfied since per-platform artifacts are independent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CdacDumpTests, CdacStressTests, CdacXPlatDumpGen, and CdacXPlatDumpTests now download the per-platform CdacBuildArtifacts_<plat> tar published by CdacBuild instead of each running their own full clr+libs+tools.cdac+... build. Each test leg still calls ./build.sh with a minimal subset (tools.cdac<test>tests) so .dotnet is initialized and MSBuild can re-link the test csproj against the downloaded artifact -- no coreclr or libs work is repeated. Adds a cdacTestConfig parameter to prepare-cdac-helix-steps.yml (default $(_BuildConfig) for back-compat); dump-test stages pass cdacTestConfig: Debug so the dotnet build invocations that build the debuggees and prepare the Helix payload pull in the Debug-configured managed cDAC assemblies that the DumpTests project loads directly. The stress stage keeps the Release native cDAC shim that ships in the shared testhost. Cross-platform job-name expansion uses dependsOnGlobalBuilds rather than a hand-rolled build_<plat>_<config>_CdacBuild string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…elism Moves CdacBuild + CdacDumpTests + CdacStressTests + CdacXPlatDumpGen into a single `Cdac` stage. Within that stage, every test job uses job-level `dependsOnGlobalBuilds: - nameSuffix: CdacBuild` which the global-build-job template expands to a per-platform `build_<plat>_<config>_CdacBuild` dep. This is the canonical dotnet/runtime pattern (runtime.yml:1591,1622,1653) and means `linux_x64` test legs start as soon as `linux_x64`'s build finishes -- no waiting on slower platforms' builds. Previously each test stage had a stage-level `dependsOn: CdacBuild` which is all-or-nothing in AzDO -- every platform's tests had to wait for the slowest platform's build to finish before starting. The single-leg / xplat / stress / schedule selection moves from stage-level `${{ if }}` to job-level `${{ if }}` inside the Cdac stage. CdacXPlatDumpTests stays as a separate stage with stage-level `dependsOn: Cdac` because its cross-platform synchronization (every target platform's test needs every source platform's dumps) is intrinsic. Its previous job-level `dependsOnGlobalBuilds: CdacBuild` is dropped -- AzDO doesn't support cross-stage job-level deps, and the stage-level dep already gates everything correctly. Also fixes a duplicate `parameters:` block in CdacXPlatDumpTests that was a merge artifact from the previous commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR restructures the runtime-diagnostics.yml pipeline to avoid redundant CoreCLR/Libraries builds across the cDAC dump tests, stress tests, and x-plat dump generation legs by introducing a per-platform “shared build” job that downstream per-platform jobs depend on.

Changes:

Collapses the prior multi-stage cDAC flow into a single Cdac stage with per-platform job-level dependencies on a shared CdacBuild job.
Switches downstream legs (dump tests, stress tests, x-plat dump gen/tests) to download artifacts/bin from the shared build instead of rebuilding CoreCLR+libs.
Extends prepare-cdac-helix-steps.yml with a cdacTestConfig parameter to allow rebuilding the managed cDAC/DumpTests payload in Debug during payload prep.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
eng/pipelines/runtime-diagnostics.yml	Refactors cDAC pipeline stages into a shared per-platform build plus dependent per-platform test legs; wires artifact upload/download and job-level deps.
eng/pipelines/cdac/prepare-cdac-helix-steps.yml	Adds `cdacTestConfig` parameter and uses it for payload-prep MSBuild invocations to support Debug-managed cDAC binaries.

+      - template: /eng/pipelines/common/platform-matrix.yml
+        parameters:
+          jobTemplate: /eng/pipelines/common/global-build-job.yml
+          buildConfig: release
+          platforms: ${{ parameters.cdacDumpPlatforms }}
+          jobParameters:


max-charlamb · 2026-06-10T16:49:16Z

Superseded by #129244 -- consolidated all the cDAC follow-up changes into a single PR.

Max Charlamb and others added 3 commits June 10, 2026 10:56

Copilot AI review requested due to automatic review settings June 10, 2026 15:06

github-actions Bot added the area-Infrastructure-coreclr Only use for closed issues label Jun 10, 2026

github-project-automation Bot added this to Runtime Infra Jun 10, 2026

Copilot started reviewing on behalf of max-charlamb June 10, 2026 15:06 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Comment thread eng/pipelines/runtime-diagnostics.yml

Comment on lines +298 to +303

- template: /eng/pipelines/common/platform-matrix.yml

parameters:

jobTemplate: /eng/pipelines/common/global-build-job.yml

buildConfig: release

platforms: ${{ parameters.cdacDumpPlatforms }}

jobParameters:

This was referenced Jun 10, 2026

[cdac] Run DataGenerator tests in CI alongside cDAC unit tests #129238

Closed

[cdac] Pipeline refactor + StackWalk doc cleanup + DataGenerator build fix #129244

Open

max-charlamb closed this Jun 10, 2026

github-project-automation Bot moved this to Done in Runtime Infra Jun 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[cdac] Share one coreclr build between cDAC dump + stress test legs#129237

[cdac] Share one coreclr build between cDAC dump + stress test legs#129237
max-charlamb wants to merge 3 commits into
dotnet:mainfrom
max-charlamb:cdac-pipeline-share-build

max-charlamb commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

max-charlamb commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

max-charlamb commented Jun 10, 2026

Summary

Before

After

How the Release / Debug cDAC split works

Files changed

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

max-charlamb commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants