[cdac] Stress framework cleanup + review feedback

Code changes:
* Eliminate fixed-size caps in the comparator pipeline:
  - MAX_COLLECTED_REFS -> SArray<StackRef> in the runtime promote
    callback, with EX_TRY/EX_CATCH around Append so an OOM doesn't
    escape the stack-walker callback's NOTHROW contract.
  - MAX_GROUPS / MAX_FRAMES -> SArray<FrameRefGroup> / SArray<FrameResult>.
  - MAX_REFS_PER_FRAME -> per-frame dispositions stored out-of-band in
    a shared SArray<RefDisposition>; cUsed/rUsed use NewArrayHolder.
    FrameResult drops the embedded 256-entry arrays (~512 bytes -> 80).
* CollectCdacStackRefs / CollectRuntimeStackRefs return HRESULT (was
  bool + side-channel logging / out-param overflow flag). Caller logs
  the hr in the [FAIL] line.
* Treat any cDAC/runtime collection failure (FAIL hr or RT OOM) as a
  hard [FAIL] rather than silently continuing with partial data.

Helix payload + harness:
* cdac-stress-helix.proj: drop the per-framework-version for-loop and
  CORE_ROOT/chmod plumbing. The Helix command is now a single
  `dotnet exec xunit.console.dll StressTests.dll` matching the
  cdac-dump-helix pattern.
* CdacStressTestBase.GetCoreRoot: walk HELIX_CORRELATION_PAYLOAD/
  shared/Microsoft.NETCore.App/<version>/ to discover CORE_ROOT.
* CdacStressTestBase: chmod +x corerun via File.SetUnixFileMode on
  non-Windows.

Doc fixes (from Copilot review feedback):
* Update `cDAC vs DAC vs runtime` references to `cDAC vs runtime''
  in runtime-diagnostics.yml + GcScanner.cs (runtime is the oracle).
* BasicCdacStressTests.cs: re-word `100% pass rate'' summary (we
  tolerate [KNOWN_ISSUE]); fix stale build recipe.
* CdacStressTestBase.cs: re-word `GCStress may not have triggered''
  failure message (we use DOTNET_CdacStress now, not DOTNET_GCStress).
* known-issues.md: log-format example matches what cdacstress.cpp
  actually emits.

Comment hygiene in cdacstress.cpp:
* Trim file header, CDAC_LOG/CDAC_ERR doc blocks, forward-decl
  preambles, Phase A/B/C/D inline section banners.
* Drop FilterRefs passthrough wrapper.
* Net -90 lines in cdacstress.cpp comments + dead code; no functional
  changes from this section.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: Max Charlamb

eng/pipelines/runtime-diagnostics.yml

@@ -321,7 +321,7 @@ extends:
                 condition: always()
 
     #
-    # cDAC GC Stress Tests — runs in-process cDAC vs DAC vs runtime stack-ref
+    # cDAC GC Stress Tests — runs in-process cDAC vs runtime stack-ref
     # verification at GC stress points. Independent stage with its own build
     # so its status/failures don't get conflated with the dump tests.
     #

src/coreclr/inc/clrconfigvalues.h

@@ -749,7 +749,7 @@ CONFIG_STRING_INFO(INTERNAL_PrestubHalt, W("PrestubHalt"), "")
 RETAIL_CONFIG_STRING_INFO(EXTERNAL_RestrictedGCStressExe, W("RestrictedGCStressExe"), "")
 RETAIL_CONFIG_DWORD_INFO(INTERNAL_CdacStressFailFast, W("CdacStressFailFast"), 0, "If nonzero, assert on cDAC/runtime GC ref mismatch during cDAC stress verification.")
 RETAIL_CONFIG_STRING_INFO(INTERNAL_CdacStressLogFile, W("CdacStressLogFile"), "Log file path for cDAC stress verification results.")
-RETAIL_CONFIG_DWORD_INFO(INTERNAL_CdacStress, W("CdacStress"), 0, "Enable cDAC stress verification. Bit flags: 0x1=alloc points.")
+RETAIL_CONFIG_DWORD_INFO(INTERNAL_CdacStress, W("CdacStress"), 0, "Enable cDAC stress verification. Bit flags: 0x1=alloc points, 0x200=verbose per-ref diagnostics.")
 CONFIG_DWORD_INFO(INTERNAL_ReturnSourceTypeForTesting, W("ReturnSourceTypeForTesting"), 0, "Allows returning the (internal only) source type of an IL to Native mapping for debugging purposes")
 RETAIL_CONFIG_DWORD_INFO(UNSUPPORTED_RSStressLog, W("RSStressLog"), 0, "Allows turning on logging for RS startup")
 CONFIG_DWORD_INFO(INTERNAL_SBDumpOnNewIndex, W("SBDumpOnNewIndex"), 0, "Used for Syncblock debugging. It's been a while since any of those have been used.")

src/coreclr/vm/cdacstress.cpp

@@ -332,7 +332,7 @@ private TargetPointer FindGCRefMap(TargetPointer indirection)
     /// </summary>
     /// <remarks>
     /// Not yet ported. Every call records a deferred frame so the stress harness
-    /// buckets the resulting cDAC-vs-DAC diff at this frame as a known issue
+    /// buckets the resulting cDAC-vs-runtime diff at this frame as a known issue
     /// rather than a real cDAC bug. Will be replaced with a real port once the
     /// signature- and ArgIterator-based ref enumeration lands.
     /// </remarks>

src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/StackWalk/GC/GcScanner.cs

@@ -12,11 +12,13 @@ namespace Microsoft.Diagnostics.DataContractReader.Tests.GCStress;
 
 /// <summary>
 /// Runs each debuggee app under corerun with DOTNET_CdacStress=0x001 (ALLOC)
-/// and asserts that the cDAC stack reference verification achieves 100% pass rate.
+/// and asserts that the cDAC stack reference verification produces no
+/// `[FAIL]` results. `[KNOWN_ISSUE]` verifications (where the cDAC explicitly
+/// marks a frame as deferred via `RecordDeferredFrame`) are tolerated.
 /// </summary>
 /// <remarks>
 /// Prerequisites:
-/// - Build CoreCLR native + cDAC: build.cmd -subset clr.native+tools.cdac -c Debug -rc Checked -lc Release
+/// - Build CoreCLR + cDAC (Checked): build.cmd -subset clr.runtime+tools.cdac -c Checked
 /// - Generate core_root: src\tests\build.cmd Checked generatelayoutonly /p:LibrariesConfiguration=Release
 /// - Build debuggees: dotnet build this test project
 ///

src/native/managed/cdac/tests/StressTests/BasicCdacStressTests.cs

@@ -34,6 +34,7 @@ internal async Task<CdacStressResults> RunGCStressAsync(string debuggeeName, int
         string coreRoot = GetCoreRoot();
         string corerun = Path.Combine(coreRoot, OperatingSystem.IsWindows() ? "corerun.exe" : "corerun");
         Assert.True(File.Exists(corerun), $"corerun not found at '{corerun}'");
+
         string debuggeeDll = GetDebuggeePath(debuggeeName);
         // When running on Helix, write logs into HELIX_WORKITEM_UPLOAD_ROOT so
         // they're uploaded as work-item artifacts and visible via the Helix API.
@@ -123,7 +124,8 @@ internal static void AssertAllPassed(CdacStressResults results, string debuggeeN
     {
         Assert.True(results.TotalVerifications > 0,
             $"GC stress test '{debuggeeName}' produced zero verifications — " +
-            "GCStress may not have triggered or cDAC may not be loaded.");
+            "the cDAC stress framework may not be enabled (DOTNET_CdacStress unset, " +
+            "or coreclr built without CDAC_STRESS).");
 
         if (results.Failed > 0)
         {
@@ -138,11 +140,26 @@ internal static void AssertAllPassed(CdacStressResults results, string debuggeeN
 
     private static string GetCoreRoot()
     {
-        // Explicit override wins (typical in CI / when running under Helix).
+        // Explicit override wins (typical when running locally with a custom layout).
         string? coreRoot = Environment.GetEnvironmentVariable("CORE_ROOT");
         if (!string.IsNullOrEmpty(coreRoot) && Directory.Exists(coreRoot))
             return coreRoot;
 
+        // Helix layout: testhost is unpacked under HELIX_CORRELATION_PAYLOAD and
+        // corerun lives in shared/Microsoft.NETCore.App/<version>/. Pick the
+        // first version directory; the payload should contain exactly one.
+        string? helixPayload = Environment.GetEnvironmentVariable("HELIX_CORRELATION_PAYLOAD");
+        if (!string.IsNullOrEmpty(helixPayload))
+        {
+            string frameworkRoot = Path.Combine(helixPayload, "shared", "Microsoft.NETCore.App");
+            if (Directory.Exists(frameworkRoot))
+            {
+                string? versionDir = Directory.EnumerateDirectories(frameworkRoot).FirstOrDefault();
+                if (versionDir is not null)
+                    return versionDir;
+            }
+        }
+
         // Local fallback: derive from the repo's standard artifact layout.
         string os = OperatingSystem.IsWindows() ? "windows" : OperatingSystem.IsMacOS() ? "osx" : "linux";
         string arch = RuntimeInformation.ProcessArchitecture.ToString().ToLowerInvariant();

src/native/managed/cdac/tests/StressTests/CdacStressTestBase.cs

@@ -58,11 +58,17 @@
     (matches what CdacStressTestBase.GetCoreRunPath expects).
   -->
   <Target Name="_CreateHelixWorkItems" BeforeTargets="CoreTest">
+    <!--
+      Run the xUnit suite directly. CdacStressTestBase.GetCoreRoot discovers
+      shared/Microsoft.NETCore.App/<version>/ from HELIX_CORRELATION_PAYLOAD
+      and the harness ensures corerun is executable, so no env-var setup or
+      framework-version loop is needed here.
+    -->
     <PropertyGroup Condition="'$(TargetOS)' == 'windows'">
-      <_StressTestCommand>for /D %25%25V in (%25HELIX_CORRELATION_PAYLOAD%25\shared\Microsoft.NETCore.App\*) do (set &quot;CORE_ROOT=%25%25V&quot; &amp;&amp; %25HELIX_CORRELATION_PAYLOAD%25\dotnet.exe exec --runtimeconfig %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.runtimeconfig.json --depsfile %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.deps.json %25HELIX_WORKITEM_PAYLOAD%25\tests\xunit.console.dll %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.dll -xml testResults.xml -nologo)</_StressTestCommand>
+      <_StressTestCommand>%25HELIX_CORRELATION_PAYLOAD%25\dotnet.exe exec --runtimeconfig %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.runtimeconfig.json --depsfile %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.deps.json %25HELIX_WORKITEM_PAYLOAD%25\tests\xunit.console.dll %25HELIX_WORKITEM_PAYLOAD%25\tests\Microsoft.Diagnostics.DataContractReader.StressTests.dll -xml testResults.xml -nologo</_StressTestCommand>
     </PropertyGroup>
     <PropertyGroup Condition="'$(TargetOS)' != 'windows'">
-      <_StressTestCommand>for d in $HELIX_CORRELATION_PAYLOAD/shared/Microsoft.NETCore.App/*/; do export CORE_ROOT=&quot;$d&quot;; chmod +x $CORE_ROOT/corerun; $HELIX_CORRELATION_PAYLOAD/dotnet exec --runtimeconfig $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.runtimeconfig.json --depsfile $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.deps.json $HELIX_WORKITEM_PAYLOAD/tests/xunit.console.dll $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.dll -xml testResults.xml -nologo; done</_StressTestCommand>
+      <_StressTestCommand>$HELIX_CORRELATION_PAYLOAD/dotnet exec --runtimeconfig $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.runtimeconfig.json --depsfile $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.deps.json $HELIX_WORKITEM_PAYLOAD/tests/xunit.console.dll $HELIX_WORKITEM_PAYLOAD/tests/Microsoft.Diagnostics.DataContractReader.StressTests.dll -xml testResults.xml -nologo</_StressTestCommand>
     </PropertyGroup>
 
     <ItemGroup>

src/native/managed/cdac/tests/StressTests/cdac-stress-helix.proj

@@ -16,7 +16,7 @@ When running `RunStressTests.ps1` (Checked, `DOTNET_CdacStress=0x001` =
 | `[FAIL]` | A real cDAC vs runtime discrepancy, or `GetStackReferences` failed at the API boundary. Investigate. |
 
 The native harness detects the deferred-frame sentinels emitted by the
-cDAC managed code and relabels per-frame diffs as `[FRAME_KNOWN_NIE]`
+cDAC managed code and relabels per-frame diffs as `[KNOWN_NIE]`
 in the structured log.
 
 ## Open buckets
@@ -25,40 +25,40 @@ in the structured log.
 
 `GcScanner.PromoteCallerStack` (in
 `src/native/managed/cdac/.../Contracts/StackWalk/GC/GcScanner.cs`)
-throws `NotImplementedException` deliberately. Producing correct
+is deliberately stubbed: instead of enumerating the caller's argument
+refs it records the frame as deferred and returns. Producing correct
 caller-argument layouts requires porting `ArgIterator` behind the
 `ICallingConvention` contract, which is a separate deferred work item.
 
 To prevent these deferred frames from masquerading as real cDAC bugs,
 the managed code records each deferred frame on the `GcScanContext`
 via `RecordDeferredFrame`, which emits a sentinel `StackRefData` entry
 with `GcScanFlags.CDAC_DEFERRED_FRAME` (0x40000000) set. The native
-stress harness strips these sentinels and re-classifies any DAC-only
-diff at a deferred Source address as `[FRAME_KNOWN_NIE]`, and the
+stress harness strips these sentinels and re-classifies any RT-only
+diff at a deferred Source address as `[KNOWN_NIE]`, and the
 whole verification as `[KNOWN_ISSUE]` rather than `[FAIL]`.
 
 Expected pattern in the log:
 
 ```
-[KNOWN_ISSUE] Thread=0x... IP=0x... cDAC=6 RT=7 (deferred frames: 1)
-  [COMPARE cDAC-vs-RT]
-    [FRAME_KNOWN_NIE] Source=0x... (<frame 0x...>): RT=1 (PromoteCallerStack deferred)
-    [MATCH] All 6 refs matched
-  [STACK_TRACE] (cDAC=6 RT=7)
-    #0 System.AppContext.Setup(...)                                   (cDAC=2 RT=2)
+[KNOWN_ISSUE] Thread=0x... IP=0x... cDAC=6 RT=7 frames=5 (match=4 mismatch=0 known_nie=1)
+  Frame #4 <frame PrestubMethodFrame 0x...> [KNOWN_NIE] cDAC=0 RT=1 SP_cDAC=0x0 SP_RT=0x0
+      [NIE(RT)] Addr=0x... Obj=0x... Flags=0x0 Reg=-1 Off=0
+  [STACK_TRACE] (cDAC=6 RT=7 frames=5)
+    #0 System.AppContext.Setup(...) (cDAC=2 RT=2)
     ...
-    #N <frame 0x...>                                                  (cDAC=0 RT=1) <-- KNOWN_NIE
+    #4 <frame PrestubMethodFrame 0x...> (cDAC=0 RT=1) <-- KNOWN_NIE (PromoteCallerStack deferred)
 ```
 
 Every JIT frame's count matches exactly; the only discrepancy is on
 the explicit transition Frame that `PromoteCallerStack` would scan.
 
 To re-enable: implement `ICallingConvention.PortableArgumentIterator`,
-then replace the `throw` in `PromoteCallerStack` with a call into the
-new contract. Once that lands, the previously-tracked
+then replace the `RecordDeferredFrame` stub in `PromoteCallerStack` with
+a call into the new contract. Once that lands, the previously-tracked
 `ELEMENT_TYPE_INTERNAL` (0x21) case in signature decoding will also
 need to be handled — that case currently isn't reachable because
-`PromoteCallerStack` short-circuits with `NotImplementedException`.
+`PromoteCallerStack` short-circuits without iterating the signature.
 
 ## Future work
 
@@ -70,18 +70,28 @@ need to be handled — that case currently isn't reachable because
 
 ## Log Format
 
-The stress log uses structured per-frame output with method-name
-resolution:
+Each verification emits a single header line followed by, on `[FAIL]` or
+`[KNOWN_ISSUE]`, a per-broken-frame block and a stack trace.
 
 ```
-[PASS] Thread=0x... IP=0x... cDAC=N RT=N
-[FAIL] Thread=0x... IP=0x... cDAC=N RT=M
-  [COMPARE cDAC-vs-RT]
-    [FRAME_DIFF] Source=0x... (MethodName): cDAC=X RT=Y
-      [cDAC_ONLY] Addr=0x... Obj=0x... Flags=0x...
-      [RT_ONLY] Addr=0x... Obj=0x... Flags=0x...
-    [FRAME_cDAC_ONLY] Source=0x... (MethodName): cDAC=X
-    [FRAME_RT_ONLY] Source=0x... (<frame 0x...>): RT=Y
-  [STACK_TRACE] (cDAC=N RT=M)
+[PASS] Thread=0x... IP=0x... cDAC=N RT=N frames=N
+
+[KNOWN_ISSUE] Thread=0x... IP=0x... cDAC=N RT=M frames=N (match=N mismatch=N known_nie=N)
+  Frame #i <frame TypeName 0x...> [KNOWN_NIE] cDAC=X RT=Y SP_cDAC=0x... SP_RT=0x...
+      [NIE(RT)] Addr=0x... Obj=0x... Flags=0x... Reg=N Off=N
+  [STACK_TRACE] (cDAC=N RT=M frames=N)
+    #i MethodName (cDAC=X RT=Y)
+    #i <frame TypeName 0x...> (cDAC=X RT=Y) <-- KNOWN_NIE (PromoteCallerStack deferred)
+
+[FAIL] Thread=0x... IP=0x... cDAC=N RT=M frames=N (match=N mismatch=N known_nie=N)
+  Frame #i MethodName [MISMATCH] cDAC=X RT=Y SP_cDAC=0x... SP_RT=0x...
+      [ONLY(cDAC)] Addr=0x... Obj=0x... Flags=0x... Reg=N Off=N
+      [ONLY(RT)]   Addr=0x... Obj=0x... Flags=0x... Reg=N Off=N
+  [STACK_TRACE] (cDAC=N RT=M frames=N)
     #i MethodName (cDAC=X RT=Y) [<-- MISMATCH]
 ```
+
+Frames whose counts match are omitted from the per-frame block in
+concise mode; verbose mode (`DOTNET_CdacStress=0x201`) also emits the
+matched refs.
+

src/native/managed/cdac/tests/StressTests/known-issues.md

@@ -5,11 +5,11 @@ a target process without needing a real runtime.
 
 For integration tests that exercise the cDAC against a real runtime, see:
 
-- [DumpTests](DumpTests/README.md) — validates cDAC contracts against crash dumps
+- [DumpTests](../DumpTests/README.md) — validates cDAC contracts against crash dumps
   produced by purpose-built debuggees.
-- [StressTests](StressTests/README.md) — in-process GC stress verification that
+- [StressTests](../StressTests/README.md) — in-process GC stress verification that
   compares cDAC stack-reference enumeration against the runtime's own GC root
-  scanning at every GC stress trigger point.
+  scanning at every wired stress trigger point (currently managed allocation).
 
 ## Building and running