diff --git a/docs/design/datacontracts/StackWalk.md b/docs/design/datacontracts/StackWalk.md index 9ef6d502334c2e..5ca5484bc71a19 100644 --- a/docs/design/datacontracts/StackWalk.md +++ b/docs/design/datacontracts/StackWalk.md @@ -626,26 +626,7 @@ At each frame yielded by `Filter`, the walk determines whether to scan for GC re - **PrestubMethodFrame / CallCountingHelperFrame**: Use signature-based scanning. - Other frame types: No GC roots to report. -See [GCRefMap Format and Resolution](#gcrefmap-format-and-resolution) for the GCRefMap scanning path and [Signature-Based Scanning](#signature-based-scanning) for the signature decoding path. - -### Signature-Based Scanning (currently deferred) - -When a transition frame's calling convention is not described by a precomputed GCRefMap (`PrestubMethodFrame`, `CallCountingHelperFrame`, and the fallback path for `StubDispatchFrame`/`ExternalMethodFrame`), the native runtime classifies caller-stack arguments by decoding the callee's method signature (`TransitionFrame::PromoteCallerStack` in `src/coreclr/vm/frames.cpp`). - -The cDAC does **not** currently port this scan. `GcScanner.PromoteCallerStack` is a stub that records the frame as deferred and returns without enumerating any refs: - -```csharp -private static void PromoteCallerStack(TargetPointer frameAddress, GcScanContext scanContext) -{ - scanContext.RecordDeferredFrame(frameAddress); -} -``` - -`RecordDeferredFrame` (on `GcScanContext`) appends a sentinel `StackRefData` entry with `Flags = GcScanFlags.CDAC_DEFERRED_FRAME (0x40000000)` and `Source = frameAddress`. The sentinel has no real GC ref payload; downstream consumers (e.g. the cDAC stress harness in `src/coreclr/vm/cdacstress.cpp`) can detect it and treat the missing refs at that frame as expected gaps rather than cDAC bugs. See [tests/StressTests/known-issues.md](../../../src/native/managed/cdac/tests/StressTests/known-issues.md) for the stress framework's handling and the tracking work to re-enable the scan. - -The `GcSignatureTypeProvider` class remains in the tree as the scaffolding the eventual port will use; it has no callers while `PromoteCallerStack` is stubbed. - -Tracking work to re-enable the scan: it requires porting `ArgIterator` behind an `ICallingConvention` contract. Once that lands, `PromoteCallerStack` will fan out into the signature-decoding algorithm (reserved-slot computation, signature walk, slot reporting) that mirrors the native version. See also [dotnet/runtime#127765](https://github.com/dotnet/runtime/issues/127765). +See [GCRefMap Format and Resolution](#gcrefmap-format-and-resolution) for the GCRefMap scanning path. The signature-based scanning fallback (`PromoteCallerStack`) is currently a stub awaiting an `ICallingConvention` contract port; the stress harness handles the resulting gaps via a deferred-frame sentinel (see [tests/StressTests/known-issues.md](../../../src/native/managed/cdac/tests/StressTests/known-issues.md) and tracking issue [dotnet/runtime#127765](https://github.com/dotnet/runtime/issues/127765)). ### GCRefMap Format and Resolution diff --git a/eng/Subsets.props b/eng/Subsets.props index b753c4422c1d55..b618ad17a150d7 100644 --- a/eng/Subsets.props +++ b/eng/Subsets.props @@ -531,6 +531,7 @@ + diff --git a/eng/pipelines/cdac/cdac-helix-test-leg.yml b/eng/pipelines/cdac/cdac-helix-test-leg.yml new file mode 100644 index 00000000000000..f35df99f53cb35 --- /dev/null +++ b/eng/pipelines/cdac/cdac-helix-test-leg.yml @@ -0,0 +1,106 @@ +# cdac-helix-test-leg.yml -- wraps the matrix + per-platform job + artifact +# download + prepare + send-to-helix + fail-on-error boilerplate shared by +# the three cDAC dump-style legs in runtime-diagnostics.yml: +# +# CdacDumpTests (single-leg mode) -- runs DumpTests directly +# CdacXPlatDumpGen -- generates dumps only (DumpOnly=true) +# CdacXPlatDumpTests -- runs DumpTests against dumps from all source platforms +# +# All three: download the shared CdacBuild artifact, prepare a Helix payload, +# send to Helix, fail the job if Helix reports failures. They differ in: +# - which platform list they fan out over +# - which Helix .proj they send +# - extra prepare-cdac-helix-steps parameters (skipDebuggeeCopy, etc.) +# - extra postBuildSteps inserted between prepare and send (xplat-test needs +# to download all source platforms' dumps and extract the tars first) +# - whether to publish dump artifacts at the end, and on what condition +# +# Stress tests are not collapsed into this template -- they use a different +# prepare template (prepare-cdac-stress-helix-steps.yml) and have unique +# crash-dump-symbolication artifacts, so they stay inline. + +parameters: +- name: nameSuffix + type: string +- name: platforms + type: object +- name: sendDisplayName + type: string +- name: sendParams + type: string +- name: prepareParameters + type: object + default: {} +- name: dependsOnCdacBuild + type: boolean + default: true +- name: extraStepsBeforeSend + type: stepList + default: [] +- name: failDisplayName + type: string + default: 'Fail on test errors' +- name: failErrorMessage + type: string + default: 'One or more cDAC Helix failures were detected. Failing the job.' +- name: publishDumpsArtifactName + type: string + default: '' # if empty, no dump-artifact publish step is emitted +- name: publishDumpsCondition + type: string + default: 'and(always(), ne(variables[''Agent.JobStatus''], ''Succeeded''))' +- name: extraDependsOn + type: object + default: [] # explicit job names to add to dependsOn (combined with dependsOnGlobalBuilds) + +jobs: +- template: /eng/pipelines/common/platform-matrix.yml + parameters: + jobTemplate: /eng/pipelines/common/global-build-job.yml + buildConfig: release + platforms: ${{ parameters.platforms }} + shouldContinueOnError: true + jobParameters: + nameSuffix: ${{ parameters.nameSuffix }} + # Tiny rebuild that initializes .dotnet and links the test csproj + # against the downloaded CdacBuild artifact; coreclr + libs + tools.cdac + # are not rebuilt. + buildArgs: -s tools.cdacdumptests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) /p:SkipDumpVersions=net10.0 + timeoutInMinutes: 180 + ${{ if parameters.dependsOnCdacBuild }}: + dependsOnGlobalBuilds: + - nameSuffix: CdacBuild + ${{ if ne(length(parameters.extraDependsOn), 0) }}: + dependsOn: ${{ parameters.extraDependsOn }} + preBuildSteps: + - template: /eng/pipelines/cdac/download-cdac-build-artifact.yml + postBuildSteps: + - template: /eng/pipelines/cdac/prepare-cdac-helix-steps.yml + parameters: ${{ parameters.prepareParameters }} + - ${{ each step in parameters.extraStepsBeforeSend }}: + - ${{ step }} + - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml + parameters: + displayName: ${{ parameters.sendDisplayName }} + sendParams: ${{ parameters.sendParams }} + environment: + _Creator: dotnet-bot + SYSTEM_ACCESSTOKEN: $(System.AccessToken) + NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages + ${{ if eq(parameters.nameSuffix, 'CdacXPlatDumpTest') }}: + SourcePlatforms: ${{ join(';', parameters.platforms) }} + - ${{ if ne(parameters.publishDumpsArtifactName, '') }}: + - task: PublishPipelineArtifact@1 + inputs: + targetPath: $(Build.SourcesDirectory)/artifacts/helixresults + artifactName: ${{ parameters.publishDumpsArtifactName }} + displayName: 'Publish Dump Artifacts' + condition: ${{ parameters.publishDumpsCondition }} + continueOnError: true + - pwsh: | + if ("$(Agent.JobStatus)" -ne "Succeeded") { + Write-Error "${{ parameters.failErrorMessage }}" + exit 1 + } + displayName: ${{ parameters.failDisplayName }} + condition: always() diff --git a/eng/pipelines/cdac/download-cdac-build-artifact.yml b/eng/pipelines/cdac/download-cdac-build-artifact.yml new file mode 100644 index 00000000000000..23c534ce680ebe --- /dev/null +++ b/eng/pipelines/cdac/download-cdac-build-artifact.yml @@ -0,0 +1,13 @@ +# download-cdac-build-artifact.yml -- per-platform download of the +# CdacBuildArtifacts_ tarball published by the CdacBuild job. +# Extracts into artifacts/bin/, so every test leg sees the same coreclr + +# libs + cDAC + test-csproj outputs that CdacBuild produced. Used as a +# preBuildStep by every test leg in the Cdac stage (and by CdacXPlatDumpTests). + +steps: +- template: /eng/pipelines/common/download-artifact-step.yml + parameters: + artifactName: CdacBuildArtifacts_$(osGroup)$(osSubgroup)_$(archType) + artifactFileName: CdacBuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)$(archiveExtension) + unpackFolder: $(Build.SourcesDirectory)/artifacts/bin + displayName: 'cDAC Shared Build Artifacts' diff --git a/eng/pipelines/cdac/prepare-cdac-helix-steps.yml b/eng/pipelines/cdac/prepare-cdac-helix-steps.yml index c3b61807915e56..0721fc5ac65061 100644 --- a/eng/pipelines/cdac/prepare-cdac-helix-steps.yml +++ b/eng/pipelines/cdac/prepare-cdac-helix-steps.yml @@ -7,15 +7,30 @@ parameters: buildDebuggees: true skipDebuggeeCopy: false runtimeConfiguration: 'Checked' + # Configuration the cDAC managed assemblies are built at during payload + # prep. The dump tests load the managed cDAC directly via ProjectReference, + # so passing /p:Configuration=Debug here makes MSBuild walk the + # DumpTests project graph and produce Debug-built cDAC dlls in + # DumpTests's bin folder, which PrepareHelixPayload then copies into + # the Helix payload. Defaults to $(_BuildConfig) for back-compat. + cdacTestConfig: '$(_BuildConfig)' + # Libraries configuration is pinned independently of cdacTestConfig so + # an /p:Configuration=Debug override here never causes MSBuild to look + # for Debug-built libraries (which we don't ship in CI). If the libs + # are missing at this config, the dotnet build below will fail with a + # clear unresolved-reference error. + librariesConfiguration: '$(_BuildConfig)' steps: - ${{ if parameters.buildDebuggees }}: - script: $(Build.SourcesDirectory)$(dir).dotnet$(dir)dotnet$(exeExt) msbuild $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/Microsoft.Diagnostics.DataContractReader.DumpTests.csproj /t:BuildDebuggeesOnly - /p:Configuration=$(_BuildConfig) - /p:TargetArchitecture=$(archType) + /p:Configuration=${{ parameters.cdacTestConfig }} + /p:ToolsConfiguration=${{ parameters.cdacTestConfig }} + /p:LibrariesConfiguration=${{ parameters.librariesConfiguration }} /p:RuntimeConfiguration=${{ parameters.runtimeConfiguration }} + /p:TargetArchitecture=$(archType) -bl:$(Build.SourcesDirectory)/artifacts/log/BuildDebuggees.binlog displayName: 'Build Debuggees' @@ -23,7 +38,9 @@ steps: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/Microsoft.Diagnostics.DataContractReader.DumpTests.csproj /p:PrepareHelixPayload=true /p:SkipDebuggeeCopy=${{ parameters.skipDebuggeeCopy }} - /p:Configuration=$(_BuildConfig) + /p:Configuration=${{ parameters.cdacTestConfig }} + /p:ToolsConfiguration=${{ parameters.cdacTestConfig }} + /p:LibrariesConfiguration=${{ parameters.librariesConfiguration }} /p:HelixPayloadDir=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac /p:SkipDumpVersions=net10.0 -bl:$(Build.SourcesDirectory)/artifacts/log/DumpTestPayload.binlog diff --git a/eng/pipelines/diagnostics/sos-test-leg.yml b/eng/pipelines/diagnostics/sos-test-leg.yml new file mode 100644 index 00000000000000..cb54452727b54b --- /dev/null +++ b/eng/pipelines/diagnostics/sos-test-leg.yml @@ -0,0 +1,75 @@ +# sos-test-leg.yml -- wraps the matrix + job + dep + artifact download + +# standard postBuildSteps pattern used by the SOS-style test legs (cDAC, +# cDAC_no_fallback, DAC) that run on windows_x64. Each of those legs only +# differs in its `name`, `useCdac`, and `noFallback` value. +# +# AzDO tasks workaround: +# PublishTestResults v2.270.0 garbles parameterized xUnit test names +# containing dots. Unsetting AllowPtrToDetectTestRunRetryFiles restores +# v2.262.0 behavior. Tracked at microsoft/azure-pipelines-tasks#21871. + +parameters: +- name: name + type: string +- name: useCdac + type: boolean + default: false +- name: noFallback + type: boolean + default: false + +jobs: +- template: /eng/pipelines/common/platform-matrix.yml + parameters: + jobTemplate: /eng/pipelines/diagnostics/runtime-diag-job.yml + buildConfig: release + platforms: + - windows_x64 + variables: + - name: AllowPtrToDetectTestRunRetryFiles + value: false + jobParameters: + name: ${{ parameters.name }} + useCdac: ${{ parameters.useCdac }} + noFallback: ${{ parameters.noFallback }} + testInterpreter: true + methodFilter: SOS* + isOfficialBuild: ${{ variables.isOfficialBuild }} + liveRuntimeDir: $(Build.SourcesDirectory)/artifacts/runtime + timeoutInMinutes: 360 + dependsOn: + - build_windows_x64_release_AllSubsets_CoreCLR + preBuildSteps: + - template: /eng/pipelines/common/download-artifact-step.yml + parameters: + artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr + artifactFileName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr$(archiveExtension) + unpackFolder: $(Build.SourcesDirectory)/artifacts/runtime + displayName: 'Runtime Build Artifacts' + postBuildSteps: + - task: PublishTestResults@2 + inputs: + testResultsFormat: xUnit + testResultsFiles: '**/*.xml' + searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults' + testRunTitle: 'Tests $(_PhaseName)' + failTaskOnFailedTests: true + publishRunAttachments: true + mergeTestResults: true + buildConfiguration: $(_BuildConfig) + continueOnError: true + condition: always() + - task: PublishPipelineArtifact@1 + inputs: + targetPath: '$(Build.SourcesDirectory)/artifacts/tmp/$(_BuildConfig)/dumps' + artifactName: 'Dumps_${{ parameters.name }}_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' + displayName: 'Publish Crash Dumps' + continueOnError: true + condition: failed() + - task: PublishPipelineArtifact@1 + inputs: + targetPath: '$(Build.SourcesDirectory)/artifacts/TestResults' + artifactName: 'TestResults_${{ parameters.name }}_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' + displayName: 'Publish Test Results and SOS Logs' + continueOnError: true + condition: failed() diff --git a/eng/pipelines/runtime-diagnostics.yml b/eng/pipelines/runtime-diagnostics.yml index f13e62a096b366..f5fdf347715509 100644 --- a/eng/pipelines/runtime-diagnostics.yml +++ b/eng/pipelines/runtime-diagnostics.yml @@ -1,3 +1,31 @@ +# runtime-diagnostics.yml +# +# Pipeline overview: +# +# SOSTests windows_x64 Release. One shared coreclr+libs build (-c Debug +# -rc release -lc release -clrinterpreter) consumed by 3 SOS +# legs that run on top of it: cDAC, cDAC_no_fallback, DAC. +# Each leg sets testInterpreter: true so interpreter coverage +# is exercised inline (no separate Interpreter leg). +# +# CdacUnitTests linux_x64 Debug. Builds + tests the cDAC managed-side +# UnitTests + DataGeneratorTests projects (tools.cdactests +# subset). Independent of every other stage. +# +# CdacTests Per-platform CdacBuild + per-platform Dump / Stress / +# XPlatDumpGen / XPlatDumpTest legs in one stage. Each +# test leg uses job-level dependsOnGlobalBuilds: CdacBuild, +# so e.g. linux_x64 dump tests start as soon as linux_x64 +# CdacBuild finishes (no waiting on the slowest platform). +# The cross-platform XPlatDumpTest legs additionally +# depend on every source platform's XPlatDumpGen via +# explicit per-job dependsOn. +# Mode gating: +# CdacDumpTest single-leg mode AND not Schedule +# CdacStressTest always +# CdacXPlatDumpGen xplat mode OR Schedule +# CdacXPlatDumpTest xplat mode OR Schedule + trigger: none parameters: @@ -80,8 +108,18 @@ extends: template: /eng/pipelines/common/templates/pipeline-with-resources.yml parameters: stages: - - stage: Build + + # ---------------------------------------------------------------------- + # SOS tests: shared windows_x64 build + 3 SOS legs that consume it. + # ---------------------------------------------------------------------- + - stage: SOSTests jobs: + # Shared coreclr+libs+host+packs build. -clrinterpreter compiles + # FEATURE_INTERPRETER into the Release CoreCLR so the SOS legs below + # (which all set testInterpreter: true via sos-test-leg.yml) get + # interpreter coverage without needing a separate Checked drop + # (see review on dotnet/runtime#127840 -- -clrinterpreter enables + # the interpreter in Release too). - template: /eng/pipelines/common/platform-matrix.yml parameters: jobTemplate: /eng/pipelines/common/global-build-job.yml @@ -89,11 +127,6 @@ extends: platforms: - windows_x64 jobParameters: - # Pass -clrinterpreter so FEATURE_INTERPRETER is compiled into the Release - # CoreCLR. This lets the Interpreter SOS leg below share a single build with - # the cDAC/cDAC_no_fallback/DAC legs instead of producing a separate Checked - # drop. (Note from review on dotnet/runtime#127840: -clrinterpreter enables - # the interpreter in Release as well.) buildArgs: -s clr+libs+tools.cdac+host+packs -c Debug -rc $(_BuildConfig) -lc $(_BuildConfig) -clrinterpreter nameSuffix: AllSubsets_CoreCLR isOfficialBuild: ${{ variables.isOfficialBuild }} @@ -112,341 +145,190 @@ extends: tarCompression: $(tarCompression) artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr displayName: Build Assets + + # SOS test legs: each depends on the shared build above. Differ only in + # name (artifact namespace) and the cDAC fallback flags. + - template: /eng/pipelines/diagnostics/sos-test-leg.yml + parameters: + name: cDAC + useCdac: true + + - template: /eng/pipelines/diagnostics/sos-test-leg.yml + parameters: + name: cDAC_no_fallback + useCdac: true + noFallback: true + + - template: /eng/pipelines/diagnostics/sos-test-leg.yml + parameters: + name: DAC + useCdac: false + + # ---------------------------------------------------------------------- + # cDAC managed-side unit tests + DataGenerator tests. Standalone, no + # native runtime needed; runs every trigger. + # ---------------------------------------------------------------------- + - stage: CdacUnitTests + dependsOn: [] + jobs: - template: /eng/pipelines/common/platform-matrix.yml parameters: - jobTemplate: /eng/pipelines/diagnostics/runtime-diag-job.yml - buildConfig: release + jobTemplate: /eng/pipelines/common/global-build-job.yml + buildConfig: debug platforms: - - windows_x64 - # Workaround: microsoft/azure-pipelines-tasks#21871 - # PublishTestResults v2.270.0 garbles parameterized xUnit test names containing dots. - # Unsetting this feature flag restores v2.262.0 behavior until a permanent fix ships. - variables: - - name: AllowPtrToDetectTestRunRetryFiles - value: false + - linux_x64 jobParameters: - name: cDAC - useCdac: true - testInterpreter: true - methodFilter: SOS* - isOfficialBuild: ${{ variables.isOfficialBuild }} - liveRuntimeDir: $(Build.SourcesDirectory)/artifacts/runtime - timeoutInMinutes: 360 - dependsOn: - - build_windows_x64_release_AllSubsets_CoreCLR - preBuildSteps: - - template: /eng/pipelines/common/download-artifact-step.yml - parameters: - artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr - artifactFileName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr$(archiveExtension) - unpackFolder: $(Build.SourcesDirectory)/artifacts/runtime - displayName: 'Runtime Build Artifacts' - postBuildSteps: - - task: PublishTestResults@2 - inputs: - testResultsFormat: xUnit - testResultsFiles: '**/*.xml' - searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults' - testRunTitle: 'Tests $(_PhaseName)' - failTaskOnFailedTests: true - publishRunAttachments: true - mergeTestResults: true - buildConfiguration: $(_BuildConfig) - continueOnError: true - condition: always() - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/tmp/$(_BuildConfig)/dumps' - artifactName: 'Dumps_cDAC_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Crash Dumps' - continueOnError: true - condition: failed() - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/TestResults' - artifactName: 'TestResults_cDAC_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Test Results and SOS Logs' - continueOnError: true - condition: failed() + nameSuffix: CdacUnitTests + # tools.cdactests subset builds both the UnitTests and + # DataGeneratorTests projects (wired in eng/Subsets.props). -test + # runs them through their xunit harness. + buildArgs: -s tools.cdactests -c $(_BuildConfig) -test + timeoutInMinutes: 120 + enablePublishTestResults: true + testResultsFormat: 'xunit' + + # ---------------------------------------------------------------------- + # cDAC dump + stress + xplat-dump-gen tests. + # + # All three test families live in this one stage so they can use + # job-level dependsOnGlobalBuilds against the shared CdacBuild jobs + # (also in this stage) -- linux_x64 test legs start as soon as + # linux_x64 CdacBuild finishes, no waiting on slower platforms. + # + # Runtime is built Checked. The cDAC native shim ships in the Release + # testhost CdacBuild publishes -- stress tests load it in-process. + # Dump tests load the managed cDAC assemblies directly and rebuild them + # at /p:Configuration=Debug during payload prep. + # ---------------------------------------------------------------------- + - stage: CdacTests + dependsOn: [] + jobs: + # Shared per-platform build. Downstream jobs in this stage depend on + # the per-platform build__release_CdacBuild via job-level + # dependsOnGlobalBuilds: - nameSuffix: CdacBuild. - template: /eng/pipelines/common/platform-matrix.yml parameters: - jobTemplate: /eng/pipelines/diagnostics/runtime-diag-job.yml + jobTemplate: /eng/pipelines/common/global-build-job.yml buildConfig: release - platforms: - - windows_x64 - # Workaround: microsoft/azure-pipelines-tasks#21871 - variables: - - name: AllowPtrToDetectTestRunRetryFiles - value: false + platforms: ${{ parameters.cdacDumpPlatforms }} jobParameters: - name: cDAC_no_fallback - useCdac: true - noFallback: true - testInterpreter: true - methodFilter: SOS* + nameSuffix: CdacBuild + buildArgs: -s clr+libs+tools.cdac+tools.cdacdumptests+tools.cdacstresstests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) /p:SkipDumpVersions=net10.0 isOfficialBuild: ${{ variables.isOfficialBuild }} - liveRuntimeDir: $(Build.SourcesDirectory)/artifacts/runtime - timeoutInMinutes: 360 - dependsOn: - - build_windows_x64_release_AllSubsets_CoreCLR - preBuildSteps: - - template: /eng/pipelines/common/download-artifact-step.yml - parameters: - artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr - artifactFileName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr$(archiveExtension) - unpackFolder: $(Build.SourcesDirectory)/artifacts/runtime - displayName: 'Runtime Build Artifacts' + timeoutInMinutes: 180 postBuildSteps: - - task: PublishTestResults@2 - inputs: - testResultsFormat: xUnit - testResultsFiles: '**/*.xml' - searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults' - testRunTitle: 'Tests $(_PhaseName)' - failTaskOnFailedTests: true - publishRunAttachments: true - mergeTestResults: true - buildConfiguration: $(_BuildConfig) - continueOnError: true - condition: always() - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/tmp/$(_BuildConfig)/dumps' - artifactName: 'Dumps_cDAC_no_fallback_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Crash Dumps' - continueOnError: true - condition: failed() - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/TestResults' - artifactName: 'TestResults_cDAC_no_fallback_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Test Results and SOS Logs' - continueOnError: true - condition: failed() + - template: /eng/pipelines/common/upload-artifact-step.yml + parameters: + rootFolder: $(Build.SourcesDirectory)/artifacts/bin + includeRootFolder: false + archiveType: $(archiveType) + archiveExtension: $(archiveExtension) + tarCompression: $(tarCompression) + artifactName: CdacBuildArtifacts_$(osGroup)$(osSubgroup)_$(archType) + displayName: cDAC Shared Build Artifacts + + # cDAC Dump Tests -- single-leg mode, not on schedule. + - ${{ if and(eq(parameters.cdacDumpTestMode, 'single-leg'), ne(variables['Build.Reason'], 'Schedule')) }}: + - template: /eng/pipelines/cdac/cdac-helix-test-leg.yml + parameters: + nameSuffix: CdacDumpTest + platforms: ${{ parameters.cdacDumpPlatforms }} + prepareParameters: + cdacTestConfig: Debug + sendDisplayName: 'Send cDAC Dump Tests to Helix' + sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:DumpTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac /bl:$(Build.SourcesDirectory)/artifacts/log/SendToHelix.binlog + failErrorMessage: 'One or more cDAC dump test failures were detected. Failing the job.' + publishDumpsArtifactName: CdacDumps_$(osGroup)_$(archType) + + # cDAC GC Stress Tests. Inline (uses prepare-cdac-stress-helix-steps.yml + # rather than the dump-side prepare template, and adds a unique testhost + # artifact for crash-dump symbolication). - template: /eng/pipelines/common/platform-matrix.yml parameters: - jobTemplate: /eng/pipelines/diagnostics/runtime-diag-job.yml + jobTemplate: /eng/pipelines/common/global-build-job.yml buildConfig: release - platforms: - - windows_x64 - # Workaround: microsoft/azure-pipelines-tasks#21871 - variables: - - name: AllowPtrToDetectTestRunRetryFiles - value: false + platforms: ${{ parameters.cdacStressPlatforms }} + shouldContinueOnError: true jobParameters: - name: DAC - useCdac: false - testInterpreter: true - methodFilter: SOS* - isOfficialBuild: ${{ variables.isOfficialBuild }} - liveRuntimeDir: $(Build.SourcesDirectory)/artifacts/runtime - timeoutInMinutes: 360 - dependsOn: - - build_windows_x64_release_AllSubsets_CoreCLR + nameSuffix: CdacStressTest + buildArgs: -s tools.cdacstresstests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) + timeoutInMinutes: 180 + dependsOnGlobalBuilds: + - nameSuffix: CdacBuild preBuildSteps: - - template: /eng/pipelines/common/download-artifact-step.yml - parameters: - artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr - artifactFileName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_coreclr$(archiveExtension) - unpackFolder: $(Build.SourcesDirectory)/artifacts/runtime - displayName: 'Runtime Build Artifacts' + - template: /eng/pipelines/cdac/download-cdac-build-artifact.yml postBuildSteps: - - task: PublishTestResults@2 - inputs: - testResultsFormat: xUnit - testResultsFiles: '**/*.xml' - searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults' - testRunTitle: 'Tests $(_PhaseName)' - failTaskOnFailedTests: true - publishRunAttachments: true - mergeTestResults: true - buildConfiguration: $(_BuildConfig) - continueOnError: true + - template: /eng/pipelines/cdac/prepare-cdac-stress-helix-steps.yml + - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml + parameters: + displayName: 'Send cDAC Stress Tests to Helix' + sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/StressTests/cdac-stress-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacStressHelixQueue)" /p:TestHostPayload=$(StressTestHostRootDir) /p:StressTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac-stress /bl:$(Build.SourcesDirectory)/artifacts/log/SendStressToHelix.binlog + environment: + _Creator: dotnet-bot + SYSTEM_ACCESSTOKEN: $(System.AccessToken) + NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages + - pwsh: | + if ("$(Agent.JobStatus)" -ne "Succeeded") { + Write-Error "One or more cDAC stress test failures were detected. Failing the job." + exit 1 + } + displayName: 'Fail on test errors' condition: always() + # On failure, publish the testhost so the crash dumps Helix + # collects can be symbolicated (needs libcoreclr.so, + # mscordaccore_universal, corerun + their .dbg/.pdb side files). - task: PublishPipelineArtifact@1 inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/tmp/$(_BuildConfig)/dumps' - artifactName: 'Dumps_DAC_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Crash Dumps' + targetPath: '$(StressTestHostRootDir)' + artifactName: 'TestHost_CdacStress_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' + displayName: 'Publish TestHost for crash dump symbolication' continueOnError: true condition: failed() - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(Build.SourcesDirectory)/artifacts/TestResults' - artifactName: 'TestResults_DAC_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish Test Results and SOS Logs' - continueOnError: true - condition: failed() - - # - # cDAC Dump Tests — Build, generate dumps, and run tests on Helix (single-leg mode) - # - - ${{ if and(eq(parameters.cdacDumpTestMode, 'single-leg'), ne(variables['Build.Reason'], 'Schedule')) }}: - - stage: CdacDumpTests - dependsOn: [] - jobs: - - template: /eng/pipelines/common/platform-matrix.yml - parameters: - jobTemplate: /eng/pipelines/common/global-build-job.yml - buildConfig: release - platforms: ${{ parameters.cdacDumpPlatforms }} - shouldContinueOnError: true - jobParameters: - nameSuffix: CdacDumpTest - buildArgs: -s clr+libs+tools.cdac+tools.cdacdumptests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) /p:SkipDumpVersions=net10.0 - timeoutInMinutes: 180 - postBuildSteps: - - template: /eng/pipelines/cdac/prepare-cdac-helix-steps.yml - - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml - parameters: - displayName: 'Send cDAC Dump Tests to Helix' - sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:DumpTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac /bl:$(Build.SourcesDirectory)/artifacts/log/SendToHelix.binlog - environment: - _Creator: dotnet-bot - SYSTEM_ACCESSTOKEN: $(System.AccessToken) - NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages - - task: PublishPipelineArtifact@1 - inputs: - targetPath: $(Build.SourcesDirectory)/artifacts/helixresults - artifactName: CdacDumps_$(osGroup)_$(archType) - displayName: 'Publish Dump Artifacts' - condition: and(always(), ne(variables['Agent.JobStatus'], 'Succeeded')) - continueOnError: true - - pwsh: | - if ("$(Agent.JobStatus)" -ne "Succeeded") { - Write-Error "One or more cDAC dump test failures were detected. Failing the job." - exit 1 - } - displayName: 'Fail on test errors' - condition: always() - - # - # cDAC GC Stress Tests — runs in-process cDAC vs runtime stack-ref - # verification at GC stress points. Independent stage with its own build - # so its status/failures don't get conflated with the dump tests. - # - - ${{ if ne(variables['Build.Reason'], 'Schedule') }}: - - stage: CdacStressTests - dependsOn: [] - jobs: - - template: /eng/pipelines/common/platform-matrix.yml - parameters: - jobTemplate: /eng/pipelines/common/global-build-job.yml - buildConfig: release - platforms: ${{ parameters.cdacStressPlatforms }} - shouldContinueOnError: true - jobParameters: - nameSuffix: CdacStressTest - buildArgs: -s clr+libs+tools.cdac+tools.cdacstresstests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) - timeoutInMinutes: 180 - postBuildSteps: - - template: /eng/pipelines/cdac/prepare-cdac-stress-helix-steps.yml - - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml - parameters: - displayName: 'Send cDAC Stress Tests to Helix' - sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/StressTests/cdac-stress-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacStressHelixQueue)" /p:TestHostPayload=$(StressTestHostRootDir) /p:StressTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac-stress /bl:$(Build.SourcesDirectory)/artifacts/log/SendStressToHelix.binlog - environment: - _Creator: dotnet-bot - SYSTEM_ACCESSTOKEN: $(System.AccessToken) - NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages - - pwsh: | - if ("$(Agent.JobStatus)" -ne "Succeeded") { - Write-Error "One or more cDAC stress test failures were detected. Failing the job." - exit 1 - } - displayName: 'Fail on test errors' - condition: always() - # On failure, publish the binaries needed to symbolicate the - # core dumps Helix collects automatically. Without these the - # dumps are unreadable -- libcoreclr.so, mscordaccore_universal, - # corerun and their .dbg/.pdb side files are required. - - task: PublishPipelineArtifact@1 - inputs: - targetPath: '$(StressTestHostRootDir)' - artifactName: 'TestHost_CdacStress_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)_Attempt$(System.JobAttempt)' - displayName: 'Publish TestHost for crash dump symbolication' - continueOnError: true - condition: failed() - # - # cDAC X-Plat Dump Generation and Testing — Two-stage flow: - # 1. Generate dumps on each platform via Helix, download and publish as artifacts - # 2. Download all platforms' dumps and run tests on each target platform - # - - ${{ if or(eq(parameters.cdacDumpTestMode, 'xplat'), eq(variables['Build.Reason'], 'Schedule')) }}: - - stage: CdacXPlatDumpGen - dependsOn: [] - jobs: - - template: /eng/pipelines/common/platform-matrix.yml + # cDAC X-Plat Dump Generation -- xplat mode or schedule. Same template + # as the single-leg dump tests, plus /p:DumpOnly=true so Helix only + # generates and returns dumps (no in-process tests). + - ${{ if or(eq(parameters.cdacDumpTestMode, 'xplat'), eq(variables['Build.Reason'], 'Schedule')) }}: + - template: /eng/pipelines/cdac/cdac-helix-test-leg.yml parameters: - jobTemplate: /eng/pipelines/common/global-build-job.yml - buildConfig: release + nameSuffix: CdacXPlatDumpGen platforms: ${{ parameters.cdacXPlatDumpPlatforms }} - shouldContinueOnError: true - jobParameters: - nameSuffix: CdacXPlatDumpGen - buildArgs: -s clr+libs+tools.cdac+tools.cdacdumptests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) /p:SkipDumpVersions=net10.0 - timeoutInMinutes: 180 - postBuildSteps: - - template: /eng/pipelines/cdac/prepare-cdac-helix-steps.yml - - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml - parameters: - displayName: 'Send cDAC Dump Gen to Helix' - sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-helix.proj /t:Test /p:DumpOnly=true /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:DumpTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac /bl:$(Build.SourcesDirectory)/artifacts/log/SendDumpGenToHelix.binlog - environment: - _Creator: dotnet-bot - SYSTEM_ACCESSTOKEN: $(System.AccessToken) - NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages - # After Helix completes and DownloadFilesFromResults runs, publish the dump tar - - pwsh: | - $resultsDir = "$(Build.SourcesDirectory)/artifacts/helixresults" - Write-Host "Helix results directory contents:" - if (Test-Path $resultsDir) { - Get-ChildItem -Recurse $resultsDir | Select-Object -ExpandProperty FullName | ForEach-Object { Write-Host " $_" } - } else { - Write-Host " Directory not found: $resultsDir" - } - displayName: 'List Helix Results' - - task: PublishPipelineArtifact@1 - inputs: - targetPath: $(Build.SourcesDirectory)/artifacts/helixresults - artifactName: CdacDumps_$(osGroup)_$(archType) - displayName: 'Publish Dump Artifacts' - - pwsh: | - if ("$(Agent.JobStatus)" -ne "Succeeded") { - Write-Error "One or more cDAC dump generation failures were detected. Failing the job." - exit 1 - } - displayName: 'Fail on dump generation errors' - condition: always() + prepareParameters: + cdacTestConfig: Debug + sendDisplayName: 'Send cDAC Dump Gen to Helix' + sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-helix.proj /t:Test /p:DumpOnly=true /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:DumpTestsPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac /bl:$(Build.SourcesDirectory)/artifacts/log/SendDumpGenToHelix.binlog + failDisplayName: 'Fail on dump generation errors' + failErrorMessage: 'One or more cDAC dump generation failures were detected. Failing the job.' + publishDumpsArtifactName: CdacDumps_$(osGroup)_$(archType) + publishDumpsCondition: 'always()' - - stage: CdacXPlatDumpTests - dependsOn: - - CdacXPlatDumpGen - jobs: - - template: /eng/pipelines/common/platform-matrix.yml + # cDAC X-Plat Dump Tests -- runs DumpTests against every source platform's + # dumps. Cross-platform fan-in: each target's test depends on every source + # platform's CdacXPlatDumpGen via explicit per-job dependsOn (platform + # names follow the build__release_ convention). Stays in + # this stage so it starts as soon as all xplat-gens are done, without + # waiting on stress tests in the same stage. + - ${{ if or(eq(parameters.cdacDumpTestMode, 'xplat'), eq(variables['Build.Reason'], 'Schedule')) }}: + - template: /eng/pipelines/cdac/cdac-helix-test-leg.yml parameters: - jobTemplate: /eng/pipelines/common/global-build-job.yml - buildConfig: release + nameSuffix: CdacXPlatDumpTest platforms: ${{ parameters.cdacXPlatDumpPlatforms }} - shouldContinueOnError: true - jobParameters: - nameSuffix: CdacXPlatDumpTest - buildArgs: -s clr+libs+tools.cdac+tools.cdacdumptests -c $(_BuildConfig) -rc checked -lc $(_BuildConfig) /p:SkipDumpVersions=net10.0 - timeoutInMinutes: 180 - postBuildSteps: - - template: /eng/pipelines/cdac/prepare-cdac-helix-steps.yml - parameters: - buildDebuggees: false - skipDebuggeeCopy: true - # Download dump artifacts from all source platforms + extraDependsOn: + - ${{ each src in parameters.cdacXPlatDumpPlatforms }}: + - build_${{ src }}_release_CdacXPlatDumpGen + prepareParameters: + buildDebuggees: false + skipDebuggeeCopy: true + cdacTestConfig: Debug + extraStepsBeforeSend: + # Download dump artifacts from all source platforms. - ${{ each platform in parameters.cdacXPlatDumpPlatforms }}: - task: DownloadPipelineArtifact@2 inputs: artifactName: CdacDumps_${{ platform }} targetPath: $(Build.SourcesDirectory)/artifacts/xplatDumps/${{ platform }} displayName: 'Download dumps from ${{ platform }}' - # Extract dump tars into the Helix payload + # Extract dump tars into the Helix payload. - pwsh: | $platforms = "${{ join(';', parameters.cdacXPlatDumpPlatforms) }}".Split(';') $payloadDumpsDir = "$(Build.SourcesDirectory)/artifacts/helixPayload/cdac/dumps" @@ -465,19 +347,6 @@ extends: Write-Host "Dump payload contents:" Get-ChildItem -Recurse $payloadDumpsDir -Filter "*.dmp" | Select-Object -ExpandProperty FullName | ForEach-Object { Write-Host " $_" } displayName: 'Extract Dump Artifacts into Payload' - - template: /eng/pipelines/common/templates/runtimes/send-to-helix-inner-step.yml - parameters: - displayName: 'Send cDAC X-Plat Dump Tests to Helix' - sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-xplat-test-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:TestPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac/tests /p:DumpPayloadBase=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac/dumps /bl:$(Build.SourcesDirectory)/artifacts/log/SendXPlatTestToHelix.binlog - environment: - _Creator: dotnet-bot - SYSTEM_ACCESSTOKEN: $(System.AccessToken) - NUGET_PACKAGES: $(Build.SourcesDirectory)$(dir).packages - SourcePlatforms: ${{ join(';', parameters.cdacXPlatDumpPlatforms) }} - - pwsh: | - if ("$(Agent.JobStatus)" -ne "Succeeded") { - Write-Error "One or more cDAC x-plat dump test failures were detected. Failing the job." - exit 1 - } - displayName: 'Fail on test errors' - condition: always() + sendDisplayName: 'Send cDAC X-Plat Dump Tests to Helix' + sendParams: $(Build.SourcesDirectory)/src/native/managed/cdac/tests/DumpTests/cdac-dump-xplat-test-helix.proj /t:Test /p:TargetOS=$(osGroup) /p:TargetArchitecture=$(archType) /p:HelixTargetQueues="$(CdacHelixQueue)" /p:TestHostPayload=$(TestHostPayloadDir) /p:TestPayload=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac/tests /p:DumpPayloadBase=$(Build.SourcesDirectory)/artifacts/helixPayload/cdac/dumps /bl:$(Build.SourcesDirectory)/artifacts/log/SendXPlatTestToHelix.binlog + failErrorMessage: 'One or more cDAC x-plat dump test failures were detected. Failing the job.' diff --git a/src/native/managed/cdac/cdac.slnx b/src/native/managed/cdac/cdac.slnx index 9828e862cc3cb6..bd5163b215640c 100644 --- a/src/native/managed/cdac/cdac.slnx +++ b/src/native/managed/cdac/cdac.slnx @@ -16,6 +16,6 @@ - + diff --git a/src/native/managed/cdac/tests/DataGenerator/Microsoft.Diagnostics.DataContractReader.DataGeneratorTests.csproj b/src/native/managed/cdac/tests/DataGenerator/Microsoft.Diagnostics.DataContractReader.DataGenerator.Tests.csproj similarity index 100% rename from src/native/managed/cdac/tests/DataGenerator/Microsoft.Diagnostics.DataContractReader.DataGeneratorTests.csproj rename to src/native/managed/cdac/tests/DataGenerator/Microsoft.Diagnostics.DataContractReader.DataGenerator.Tests.csproj diff --git a/src/native/managed/cdac/tests/DataGenerator/TestTarget.cs b/src/native/managed/cdac/tests/DataGenerator/TestTarget.cs index e1e9a7e8bd6392..b823054d3b0220 100644 --- a/src/native/managed/cdac/tests/DataGenerator/TestTarget.cs +++ b/src/native/managed/cdac/tests/DataGenerator/TestTarget.cs @@ -199,6 +199,9 @@ public override TargetCodePointer ReadCodePointer(ulong address) public override TargetNUInt ReadNUInt(ulong address) => new TargetNUInt(PointerSize == 8 ? Read(address) : Read(address)); + public override TargetNInt ReadNInt(ulong address) + => new TargetNInt(PointerSize == 8 ? Read(address) : Read(address)); + public override void Write(ulong address, T value) { Span span = GetSpan(address, System.Runtime.CompilerServices.Unsafe.SizeOf());