Skip to content

[NO REVIEW] Move ArgIterator/TransitionBlock to tools/Common for cDAC reuse#128460

Closed
max-charlamb wants to merge 7 commits into
mainfrom
cdac-shared-argiterator
Closed

[NO REVIEW] Move ArgIterator/TransitionBlock to tools/Common for cDAC reuse#128460
max-charlamb wants to merge 7 commits into
mainfrom
cdac-shared-argiterator

Conversation

@max-charlamb

@max-charlamb max-charlamb commented May 21, 2026

Copy link
Copy Markdown
Member

Note

This PR was created with Copilot assistance.

Summary

Share crossgen2's ArgIterator and TransitionBlock with the cDAC by moving them to tools/Common/CallingConvention/, introducing an ITypeHandle interface abstraction, and building a new CallingConvention contract that enumerates GC references on caller stack frames.

Changes

Shared calling convention infrastructure (commits 1-4)

  • Move ArgIterator.cs and TransitionBlock.cs from ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/ to tools/Common/CallingConvention/
  • Introduce ITypeHandle interface -- abstraction over type info needed by ArgIterator (size, CorElementType, SystemV classification, HFA, etc.)
  • Extract crossgen2's TypeHandle struct to its own file and implement ITypeHandle
  • Remove TypeSystemContext dependency from TransitionBlock.FromTarget() -- decompose into (TargetArchitecture, isWindows, isApplePlatform, isArmel) parameters to avoid pulling in the full TypeSystem dependency chain
  • Change namespace to Internal.CallingConvention for shared consumption

cDAC CallingConvention contract (commits 5-6)

  • ICallingConvention contract with EnumerateCallerStackRefs API
  • CallingConvention_1 implementation -- uses the shared ArgIterator to walk arguments and report GC references (object refs, byrefs, and struct fields via GCDesc)
  • CdacTypeHandle adapter -- implements ITypeHandle backed by the cDAC's IRuntimeTypeSystem; throws NotImplementedException for unimplemented platform-specific callbacks (SystemV struct-in-registers, HFA, FP structs, x86 trivial structs)
  • EnumerateValueTypeGCRefs -- walks GCDesc series to enumerate GC pointers within unboxed value types on the stack
  • 10 unit tests for GCDesc-based value type GC reference enumeration
  • Scope: AMD64 Windows only; other platforms will throw NotImplementedException

File-link structure

tools/Common/CallingConvention/
  ArgIterator.cs          (shared - file-linked into crossgen2 and cDAC)
  TransitionBlock.cs      (shared - file-linked into crossgen2 and cDAC)
  ITypeHandle.cs          (shared interface)
  FpStructInRegistersInfo.cs
  SystemVAmd64PassingDescriptor.cs

ILCompiler.ReadyToRun/
  TypeHandle.cs           (crossgen2-specific ITypeHandle impl)

cDAC Contracts/CallingConvention/
  CallingConvention_1.cs  (contract implementation)
  CdacTypeHandle.cs       (cDAC-specific ITypeHandle impl)

Copilot AI review requested due to automatic review settings May 21, 2026 21:10
@github-actions github-actions Bot added the area-crossgen2-coreclr only use for closed issues label May 21, 2026
@max-charlamb max-charlamb added the NO-REVIEW Experimental/testing PR, do NOT review it label May 21, 2026
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR relocates the ReadyToRun calling-convention helpers (ArgIterator/TransitionBlock) into src/coreclr/tools/Common/CallingConvention/ and updates ILCompiler.ReadyToRun.csproj to consume them via Link includes, enabling shared reuse across tooling.

Changes:

  • Added ArgIterator.cs and TransitionBlock.cs under tools/Common/CallingConvention/.
  • Updated ILCompiler.ReadyToRun.csproj to link these files from the new Common location and removed the old per-project compile includes.
  • (Review note) Found a likely copy/paste bug in TransitionBlock.ComputeReturnValueTreatment for SystemV AMD64 16-byte struct return encoding (second eightbyte classification check).

Reviewed changes

Copilot reviewed 1 out of 3 changed files in this pull request and generated no comments.

File Description
src/coreclr/tools/Common/CallingConvention/ArgIterator.cs Relocated ArgIterator implementation into tools/Common for reuse.
src/coreclr/tools/Common/CallingConvention/TransitionBlock.cs Relocated TransitionBlock implementation into tools/Common for reuse (also contains a likely SystemV return-encoding bug).
src/coreclr/tools/aot/ILCompiler.ReadyToRun/ILCompiler.ReadyToRun.csproj Switched to Link-based compilation from the new Common location and removed old compile entries.

@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings May 21, 2026 22:04
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/coreclr/tools/Common/CallingConvention/ArgIterator.cs:187

  • ArgIteratorData now stores arguments/return types as ITypeHandle. Since the concrete crossgen2 implementation (TypeHandle) is a struct, assigning it into an ITypeHandle[] boxes each element, adding per-parameter allocations and virtual dispatch in what is typically a hot path during compilation.

If the intent is to keep this code reusable without regressing crossgen2 perf, consider making the calling-convention pipeline generic over TTypeHandle (e.g., where TTypeHandle : struct, ITypeHandle) so the ReadyToRun build stays allocation-free, while still allowing a separate cDAC-backed struct implementation later.

Comment on lines +43 to +47
private static readonly int[] s_elemSizes = new int[]
{
0, //ELEMENT_TYPE_END 0x0
0, //ELEMENT_TYPE_VOID 0x1
1, //ELEMENT_TYPE_BOOLEAN 0x2
Comment on lines +32 to +38
public bool Equals(TypeHandle other)
{
return _isByRef == other._isByRef && _type == other._type;
}

public override int GetHashCode() { return (int)_type.GetHashCode(); }

Comment on lines 21 to 25
// Check to see if this argument lowers to a byref on the wasm side
TypeHandle typeHandle;
ITypeHandle typeHandle;
argit.GetArgType(out typeHandle);
if (WasmLowering.LowerToAbiType(typeHandle.GetRuntimeTypeHandle()) == null)
if (WasmLowering.LowerToAbiType(((TypeHandle)typeHandle).GetRuntimeTypeHandle()) == null)
{
@max-charlamb

Copy link
Copy Markdown
Member Author

/run

@max-charlamb

Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr r2r

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@github-actions

This comment has been minimized.

@max-charlamb

Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr r2r

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/GCRefMapBuilder.cs:95

  • parameterTypes was changed to ITypeHandle[], but the stored values are new TypeHandle(...) where TypeHandle is a struct. Storing structs into an interface-typed array causes boxing/allocation per element, which could be a non-trivial regression in crossgen2 since BuildArgIterator runs per method signature.
            bool isVarArg = false;
            TypeHandle returnType = new TypeHandle(signature.ReturnType);
            ITypeHandle[] parameterTypes = new ITypeHandle[signature.Length];
            for (int parameterIndex = 0; parameterIndex < parameterTypes.Length; parameterIndex++)
            {
                parameterTypes[parameterIndex] = new TypeHandle(signature[parameterIndex]);
            }
            CallingConventions callingConventions = (hasThis ? CallingConventions.ManagedInstance : CallingConventions.ManagedStatic);

@max-charlamb

Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr r2r

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@github-actions

This comment has been minimized.

Max Charlamb and others added 4 commits May 27, 2026 14:30
Move ArgIterator.cs and TransitionBlock.cs from the ILCompiler.ReadyToRun
project directory into src/coreclr/tools/Common/CallingConvention/ to
enable future reuse by the cDAC.

This is a pure file relocation with no code changes. The namespace
remains ILCompiler.DependencyAnalysis.ReadyToRun. The csproj is
updated to use file-link includes from the new Common location.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create ITypeHandle interface in CallingConvention/ with all type
  queries needed by ArgIterator and TransitionBlock
- Extract TypeHandle struct from ArgIterator.cs into TypeHandle.cs
- Move GetElemSize helper to static method on ITypeHandle
- Replace all GetRuntimeTypeHandle() escape hatches in ArgIterator (3)
  and TransitionBlock (3) with ITypeHandle method calls
- Port all TypeHandle references in ArgIterator/TransitionBlock to
  ITypeHandle (fields, parameters, locals, return types)
- TypeHandle retains GetRuntimeTypeHandle() for crossgen2-only code
  (WasmLowering) but it is not part of the interface

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypeHandle wraps TypeDesc which is crossgen2-specific, so it belongs
in the ReadyToRun project rather than in the shared Common directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…amespace and remove TypeSystemContext dependency

- Change namespace from ILCompiler.DependencyAnalysis.ReadyToRun to
  Internal.Runtime.CallingConvention for the shared files (ArgIterator,
  TransitionBlock, ITypeHandle)
- Remove TypeSystemContext from ArgIterator constructor; pass
  TransitionBlock directly along with isWindows, objectTypeHandle,
  and intPtrTypeHandle parameters
- Update all consumer files with new using and ArgIterator alias
  to resolve ambiguity with System.ArgIterator

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Max Charlamb and others added 3 commits May 27, 2026 14:30
…peHandle adapter

- Change namespace to Internal.CallingConvention
- Replace TargetDetails with individual params in TransitionBlock.FromTarget
- Extract standalone SystemV/FpStruct types for cDAC use
- Refactor ReportPointersFromStructInRegisters to use ITypeHandle
- Add pragma suppressions for crossgen2 analyzer warnings
- Add CdacTypeHandle adapter wrapping IRuntimeTypeSystem + TypeHandle
- Add ICallingConvention contract with EnumerateCallerStackRefs
- Add CallingConvention_1 implementation using shared ArgIterator
- Wire shared CallingConvention files into cDAC project

TODO: VarArgs support in CallingConvention contract

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…umeration

- Implement EnumerateCallerStackRefs in CallingConvention_1 to walk caller
  stack frames and report GC references for object refs, byrefs, and structs
- Add EnumerateValueTypeGCRefs using GCDesc series to enumerate GC pointers
  within unboxed value types on the stack
- Scope to AMD64 Windows; throw NotImplementedException for SystemV struct-
  in-registers, HFA, FP struct, x86 trivial structs, and WASM field alignment
- Update CdacTypeHandle stubs with NotImplementedException for unimplemented
  platform-specific callbacks
- Add 10 unit tests for GCDesc-based value type GC reference enumeration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace GC-specific CallerStackGCRef/EnumerateCallerStackRefs with
  general-purpose ArgumentLocation/EnumerateArguments on ICallingConvention
- Wire GcScanner.PromoteCallerStack to use ICallingConvention.EnumerateArguments
  instead of duplicate signature-decoding logic
- Add ReportPointersFromValueType for GCDesc-based struct GC walking
- Add IsUnboxingStub to IRuntimeTypeSystem for correct value-type this
  interior pointer detection (matches native: IsValueType && !IsUnboxingStub)
- Move CdacTypeHandle.cs to CallingConvention folder
- Remove CallingConventionTests.cs (was GCDesc-specific, needs rewrite)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 27, 2026 18:32
@max-charlamb max-charlamb force-pushed the cdac-shared-argiterator branch from 5c05657 to d4256bf Compare May 27, 2026 18:32
@github-actions

Copy link
Copy Markdown
Contributor

Note

This review was generated by Copilot.

🤖 Copilot Code Review — PR #128460

Holistic Assessment

Motivation: Sharing crossgen2's ArgIterator/TransitionBlock with the cDAC is a well-motivated deduplication effort. The calling-convention logic is non-trivial and having a single source of truth avoids drift between the two consumers. The new CallingConvention contract gives the cDAC a principled way to enumerate caller-stack arguments using the same logic as crossgen2.

Approach: The ITypeHandle interface abstraction is the right seam — it cleanly separates the type-system queries needed by ArgIterator from the concrete backing store (TypeDesc vs. cDAC MethodTable reading). The file-link model via <Compile Include> with #if READYTORUN guards follows established patterns in this codebase.

Summary: ⚠️ Needs Human Review. The refactoring of shared code is well-structured and the crossgen2 side appears mechanically correct. However, there are several concerns in the new cDAC contract implementation — particularly around error handling, GCDesc offset arithmetic, and the forcedByRefParams always being empty — that a domain expert should verify.


Detailed Findings

⚠️ forcedByRefParams is always zero-initialized — may miss forced-by-ref arguments

CallingConvention_1.cs:78 passes forcedByRefParams: new bool[parameterTypes.Length] (all false). In the native runtime, forcedByRefParams is populated from CEEInfo::getMethodInfo for architectures where large structs are passed by reference. If the callee has a struct argument that the runtime forces to pass by ref (e.g., on ARM64 for structs > 16 bytes), this array should reflect that. Currently, ArgIterator will compute IsArgPassedByRef from TransitionBlock.IsArgPassedByRef(ITypeHandle), which for AMD64 checks the size, so the behavior may be correct for that arch — but for other architectures that rely on forcedByRefParams, this could lead to incorrect offset calculations.

This may be intentional given the stated AMD64-Windows-only scope, but should be explicitly documented with a comment and/or a guard that throws for non-AMD64-Windows targets when forcedByRefParams would matter.

⚠️ GCDesc offset arithmetic in ReportPointersFromValueType — verify the - pointerSize adjustment

GcScanner.cs:394: int fieldOffset = (int)seriesOffset - pointerSize;

The comment says "GCDesc offset includes the MethodTable pointer; subtract it for unboxed layout." However, in the native runtime, GCDesc series offsets are relative to the start of the object (after the ObjHeader), not from the MethodTable pointer. For an unboxed value type on the stack, the MethodTable pointer is not present, so the subtraction should be the offset of the first field relative to the object start. In CoreCLR, for a value type, the first field starts at offset sizeof(MethodTable*) from the object base. So subtracting pointerSize (== sizeof(MethodTable*)) would be correct for 64-bit, but may not be correct if sizeof(MethodTable*) differs from pointerSize in any configuration. A domain expert should confirm this is the right adjustment for all supported targets.

⚠️ CdacTypeHandle.GetSize() subtracts 2 * PointerSize from BaseSize — fragile

CdacTypeHandle.cs:54: return (int)(baseSize - (uint)(2 * PointerSize));

This assumes BaseSize = ObjHeader + MethodTable* + fields, where both ObjHeader and MethodTable* are exactly PointerSize. While true on current 64-bit targets, this derivation is somewhat fragile and undocumented in the cDAC contract surface. If there's an existing API like GetUnboxedSize or the RuntimeTypeSystem has a way to get field size directly, that would be more robust. If not, consider adding a Debug.Assert verifying the computed size is non-negative and matches expectations.

⚠️ Silent catch blocks swallow all exceptions

CallingConvention_1.cs:43-50:

try
{
    requiresInstArg = rts.GetGenericContextLoc(methodDesc) == GenericContextLoc.InstArg;
    isAsync = rts.IsAsyncMethod(methodDesc);
}
catch
{
}

This swallows all exceptions silently. If GetGenericContextLoc throws due to corrupt memory or a bug, the code proceeds with requiresInstArg = false and isAsync = false, potentially miscomputing the argument layout. Consider at minimum catching a specific exception type, or logging/asserting on unexpected exceptions. This pattern was carried over from the old GcScanner code, but the new contract should aim for better error handling.

⚠️ ITypeHandle is internal but TypeHandle struct implements it publicly — boxing concern

TypeHandle is a struct implementing ITypeHandle (an interface). Every time ArgIterator stores a TypeHandle in an ITypeHandle field (which it does for _argTypeHandle, _argTypeHandleOfByRefParam, etc.), this causes boxing allocation on the crossgen2 side. The old code used the struct directly, avoiding boxing. This is a regression for crossgen2's hot path (GCRefMapBuilder). The performance impact may be small for crossgen2 (compilation tool, not runtime), but it's worth noting.

Existing review comment from Copilot also noted that TypeHandle doesn't override Equals(object), so boxed equality falls back to reflection-based ValueType.Equals. Consider at least adding override bool Equals(object) and override int GetHashCode() to avoid this.

💡 WasmLowering.ReadyToRun.cs — unsafe cast ((TypeHandle)typeHandle)

WasmLowering.ReadyToRun.cs:27: WasmLowering.LowerToAbiType(((TypeHandle)typeHandle).GetRuntimeTypeHandle())

This unbox-cast will throw InvalidCastException with no useful message if a non-TypeHandle ITypeHandle is passed. Since this is crossgen2-only code, it's fine today, but a Debug.Assert(typeHandle is TypeHandle) before the cast would make the assumption explicit.

✅ Shared code structure is clean

The tools/Common/CallingConvention/ directory with file-linked consumption follows the established pattern (e.g., SystemVStructClassificator.cs). The #if READYTORUN guards are minimal and well-placed. The namespace change to Internal.CallingConvention is appropriate.

TransitionBlock.FromTarget decomposition is good

Replacing the TypeSystemContext dependency with (TargetArchitecture, isWindows, isApplePlatform, isArmel) parameters is the right approach for making this code consumable outside the full TypeSystem dependency chain.

✅ GcScanner refactoring is cleaner

The new GcScanner.PromoteCallerStack using ICallingConvention.EnumerateArguments is significantly cleaner than the old manual offset computation. The switch on CorElementType is more maintainable than the old GcTypeKind enum approach.

💡 Missing tests for CallingConvention_1.EnumerateArguments

The PR description mentions "10 unit tests for GCDesc-based value type GC reference enumeration" but I don't see test files in the diff. The EnumerateArguments method itself has complex logic (metadata decoding, ArgIterator walking, struct-in-register detection) that would benefit from unit tests, especially for edge cases like methods with no parameters, static vs instance methods, and methods with struct arguments.

💡 isArmel is hardcoded to false in BuildTransitionBlock

CallingConvention_1.cs:204: return TransitionBlock.FromTarget(targetArch, isWindows, isApplePlatform, isArmel: false);

This means ARM ARMEL (soft-float) ABI will never be selected. If the cDAC ever needs to diagnose ARM ARMEL targets, this will silently produce wrong results. Consider adding a comment noting this limitation, or checking if the runtime info can distinguish ARMEL.

Generated by Code Review for issue #128460 · ● 7.1M ·

@max-charlamb

Copy link
Copy Markdown
Member Author

/azp run crossgen outerloop

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

@max-charlamb

Copy link
Copy Markdown
Member Author

/azp run runtime-coreclr crossgen2 outerloop

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

// LoongArch64/Wasm alignment
int GetFieldAlignment();

private static readonly int[] s_elemSizes = new int[]
Comment on lines +22 to +31
private readonly RuntimeInfoArchitecture _arch;
private readonly RuntimeInfoOperatingSystem _os;

public CdacTypeHandle(TypeHandle typeHandle, Target target)
{
_typeHandle = typeHandle;
_target = target;
_arch = _target.Contracts.RuntimeInfo.GetTargetArchitecture();
_os = _target.Contracts.RuntimeInfo.GetTargetOperatingSystem();
}
Comment on lines 329 to +333
IRuntimeTypeSystem rts = _target.Contracts.RuntimeTypeSystem;
ICallingConvention cc = _target.Contracts.CallingConvention;
MethodDescHandle mdh = rts.GetMethodDescHandle(methodDescPtr);

MethodSignature<GcTypeKind> methodSig;
try
foreach (ArgumentLocation arg in cc.EnumerateArguments(mdh))
Comment on lines +8 to +47
/// <summary>
/// Describes the location of an argument on a caller's transition frame,
/// produced by walking the callee's method signature with ArgIterator.
/// </summary>
public readonly struct ArgumentLocation
{
/// <summary>Byte offset from the start of the transition block.</summary>
public int Offset { get; init; }

/// <summary>The CorElementType of this argument (Class, ValueType, Byref, I4, etc.).</summary>
public CorElementType ElementType { get; init; }

/// <summary>The TypeHandle for this argument's type (needed for struct GC walking).</summary>
public TypeHandle TypeHandle { get; init; }

/// <summary>True if this is the "this" pointer slot.</summary>
public bool IsThis { get; init; }

/// <summary>True if this is a value type "this" (passed as interior pointer).</summary>
public bool IsValueTypeThis { get; init; }

/// <summary>True if this slot holds a generic instantiation parameter (MethodTable* or MethodDesc*).</summary>
public bool IsParamType { get; init; }

/// <summary>True if this argument is a struct passed by reference (e.g., large struct on AMD64).</summary>
public bool IsPassedByRef { get; init; }
}

public interface ICallingConvention : IContract
{
static string IContract.Name => nameof(CallingConvention);

/// <summary>
/// Enumerate argument locations on the caller's transition frame for the given method.
/// This uses the shared ArgIterator to walk the method signature and determine
/// where each argument resides (stack offset, element type, type handle).
/// The caller is responsible for interpreting these locations for GC or other purposes.
/// </summary>
IEnumerable<ArgumentLocation> EnumerateArguments(MethodDescHandle methodDesc) => throw new System.NotImplementedException();
}
@max-charlamb

Copy link
Copy Markdown
Member Author

Closing -- outdated. Will revisit if the ArgIterator/TransitionBlock move becomes necessary for a concrete cDAC reuse case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-crossgen2-coreclr only use for closed issues NO-REVIEW Experimental/testing PR, do NOT review it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants