Move custom attribute materialization and invocation to managed code - #133658

… instead of checked OBJECTREFs (#135537)

Fixes #135535.

On a Checked build, `OBJECTREF` is a class whose copy constructor and
`operator->` validate the address they read from against the thread's
table of unprotected object references (`Thread::dangerousObjRefs`),
which `ObjectRefNew` fills and a possible-GC point marks. The
interpreter viewed its stack slots through that class:
`LOCAL_VAR(offset, OBJECTREF)` and the typed variants,
`LOCAL_VAR_ADDR(offset, OBJECTREF)` for a call's `this`, and
`InterpreterCodeManager::GetInstance`. Interpreter stack slots do not
follow the checked `OBJECTREF` lifecycle: they are reported to the GC
through the interpreter's GC info, written by pointer-sized moves and
loads, and reused by later frames. Any entry the table holds for one of
those addresses can therefore be tripped by a later, unrelated read. In
#135535 the entry comes from `GCFrame::Remove`:
`CreateCustomAttributeInstance` (managed since #133658) registers its
four argument slots, which live on the interpreter stack, and
unregistering them records their addresses; the next possible GC point
marks them; and the next interpreted virtual call whose `this` is read
from one of those addresses fails `Thread::IsObjRefValid(this)` in
`OBJECTREF::operator->`. The table is keyed by the slot address alone,
so other producers of entries (for example `GCFrame::Pop` from
`PopExplicitFrames`) can leave the same kind of stale entry; the fix
belongs with the interpreter's accesses rather than with one source of
entries.

This change replaces the interpreter's checked slot loads and stores
with raw object-pointer accesses. `LOCAL_VAR_OBJREF` converts a raw slot
load to an `OBJECTREF`, and `SET_LOCAL_VAR_OBJREF` converts an
`OBJECTREF` to a raw slot store; other sites use raw pointers directly.
These changes cover the former `OBJECTREF`, `DELEGATEREF`,
`CONTINUATIONREF` and `BASEARRAYREF` slot accesses, including the direct
casts in continuation resume and the CALLI open-virtual-delegate
workaround. `InterpreterCodeManager::GetInstance` and the debugger's
function-evaluation argument read also load raw pointers before
conversion. Native `OBJECTREF` locals retain their usual checking and
protection. The two paths that passed a slot's address as an
`OBJECTREF*` to `MethodDesc::GetMethodDescOfVirtualizedCode` (the
CALLVIRT dispatch-cache miss, and the open virtual delegate resolution
used by CALLDELEGATE and the CALLI workaround) now go through
`ResolveVirtualCallTarget`, which copies the receiver into a native
`OBJECTREF` local protected with `GCPROTECT_BEGIN` for the resolution
and passes that local; the slot itself stays reported through the
interpreter's GC info as before. In Release builds `OBJECTREF` is
`Object*`, so the slot-access conversions expand to the code the sites
had before; the virtual call target resolution now protects a native
receiver local in all builds, where it previously passed the slot's
address. `GCFrame::Remove` is unchanged (an earlier revision of this PR
removed its `_DEBUG` loop; that is reverted).

The scheduled `runtime-interpreter` pipeline hits the assert on windows
x64 checked (Pri1 runtime tests, `interpmode3_no_tiered_compilation` and
`interpmode2_no_tiered_compilation`): in
[20261009.1](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1629767)
the work items `GC`, `JIT.performance`, `Regression_o_1`, `async`,
`reflection` and `tracing` fail under `DOTNET_InterpMode=3`, and the
same minus `GC` under `DOTNET_InterpMode=2`, each at its first affected
test; every other work item of those two jobs exits 0.

### Validation

windows-x64 Checked, at the PR base, the six runtime-test groups above
built at priority 1 and run as the Helix work items run them
(`DOTNET_InterpMode` 2 and 3, `DOTNET_TieredCompilation=0`), main vs
this change:

| work item | main, mode 3 | this PR, mode 3 | main, mode 2 | this PR,
mode 2 |
|---|---|---|---|---|
| `reflection` | assert after 5 passed | 18 passed | assert after 5
passed | 18 passed |
| `async` | assert after 87 passed | 142 passed | assert after 87 passed
| 142 passed |
| `tracing` | assert after 1 passed | 41 passed | assert after 1 passed
| 41 passed |
| `Regression_o_1` | assert after 4 passed | 158 passed | assert after 4
passed | 161 passed |
| `GC` | assert after 31 passed | 106 passed | 106 passed | 106 passed |
| `JIT.performance` | assert after 34 passed | 100 passed | assert after
22 passed | 100 passed |
| repro from #135535 | asserts | `no assert`, exit 100 | asserts | `no
assert`, exit 100 |

The repro from #135535, same roots: main asserts under modes 2 and 3;
with this change it prints `no assert` and exits 100.
`JIT/interpreter/InterpreterTester` passes with this change on all three
platforms (on windows-x64 built without the native test components, with
the same result as main). On windows-x64 Checked, a control using this
change's interpexec.cpp with the CALLVIRT `this` read restored to the
checked view, and eetwain.cpp and funceval.cpp at main, asserts in the
regression test under modes 2 and 3 and in the repro under mode 3, all
with `DOTNET_TieredCompilation=0`.

On linux-x64 Checked and osx-arm64 Checked, the repro does not assert on
main. In the Linux run inspected under lldb, the recorded entries stayed
unmarked through the virtual calls and were then flushed by
`PreStubWorker`; the macOS run was not traced. With this change, both
platforms build successfully and pass the repro and
`JIT/interpreter/InterpreterTester`; the `System.Runtime.Tests`
discovery runs under `DOTNET_InterpMode=3` record zero assertions.

The repro is added as `Regressions/coreclr/GitHub_135535`,
process-isolated, with its address sweep made denser than the issue's
version. It is configuration-sensitive by nature: reaching the assert
needs a later `this` to land on one of the recorded addresses before the
thread's next prestub flush. On windows-x64 Checked at the PR base it
asserts on main under `DOTNET_InterpMode` 2 and 3, each with and without
`DOTNET_TieredCompilation=0`, and passes with this change in those four
configurations and under the JIT, mode 1, JIT stress and GC stress (17
configurations in all). On linux-x64 and osx-arm64 the test passes on
main as well as with the change. The `runtime-interpreter` pipeline's
windows x64 work items above remain the broader coverage.

> [!NOTE]
> The analysis and this change were prepared with the help of Anthropic
Claude and reviewed by OpenAI Codex, under my direction; I verified the
results above.

## Summary

Layer 3 of the stack for #123864, following #133658. The prior
func-eval-specific proposal is #126809.

This layer started as replacing the final native managed-call dispatch
with a managed UnmanagedCallersOnly entrypoint. It now moves most
debugger function-evaluation preparation, invocation, storage, and
copy-back into managed CoreLib while retaining the debugger-specific
control flow and GC-safe bootstrap in native code.

### Managed invocation and storage

- Add `Debugger.FunctionEvaluation`, a runtime-owned UCO entrypoint that
resolves the exact target method from its declaring `RuntimeType` and
rooted `IRuntimeMethodInfo`.
- Extend the shared reflection invocation emitter with debugger-specific
thunks that prepare arguments, invoke methods and constructors, copy
back writable arguments, and publish results.
- Emit `newobj` for debugger object construction instead of
preallocating an object and calling its constructor as an ordinary
method. Existing-object constructor calls remain supported, and
abstract/interface allocation is rejected.
- Keep the entire managed call-address vector registered through
`GCFrameRegistration`, using exact typed locals for memory-backed
by-value ref structs so their managed byrefs are reported by normal JIT
GC info.
- Handle object, primitive, enum, pointer, value-type, nullable,
literal, memory, register, and byref argument representations in managed
code while preserving native-endian storage and debugger copy-back
semantics.
- Align nullable behavior with the existing debugger contract: direct
value-type memory homes are mutated in place, while debugger handles are
not treated as writable nullable homes. Exact register-backed nullable
temporaries are boxed only after their MethodTable is instance-active.
- Move ordinary value-type result storage into temporary GC-described
external memory and defer boxing until after invocation and argument
copy-back. Keep byref-like results in persistent external storage and
raw byref results rooted directly in the debugger result slot.
- Create result handles in managed code only after the result is fully
prepared, and preserve object identity and updated field contents across
collections and later debugger inspection.

### Native debugger and GC integration

- Retain native method/signature validation, generic resolution, class
initialization, receiver validation inputs, register/home capture, abort
state, exception publication, completion, and result ownership.
- Capture contiguous register value-type snapshots before the first
GC-triggering managed transition and describe them with their exact type
layout.
- Root non-leaf register references and register value-type snapshots
with `ExternalMemoryHandle` registrations so collections during managed
preparation relocate their contents correctly.
- Represent external memory with a full tagged `TypeHandle`, allowing
the runtime and cDAC to distinguish inline value types, object-reference
slots, managed-byref slots, and unmanaged pointer/function-pointer
locals.
- Report each external-memory type's loader allocator during GC
scanning, keeping collectible layouts alive while external storage is
registered.
- Add dedicated GC-safe object-register copy-back and native-endian
scalar register copy-back paths.
- Remove stale native argument copies and architecture-specific packing
layers that are no longer needed, while preserving the
architecture-specific register-home logic required by debugger IPC.
- Treat the debugger UCO entrypoint as a runtime-internal reverse
transition for exception handling, and exclude its debugger-only
MethodDesc state on WebAssembly targets that do not build `debug/ee`.
- Update the cDAC ExternalMemoryHandles reader, descriptors, tests, and
authoritative contract documentation for the exact managed-local
representation.

### Correctness fixes included

- Avoid double relocation of overlapping byrefs in by-value ref structs
(#135245).
- Keep raw byref results rooted until deferred debugger completion
(#135247).
- Prevent stale non-leaf register values after a compacting GC
(#135249).
- Preserve current external-result fields across collections and later
inspection (#135251).
- Keep collectible loader allocators alive while external-memory layouts
are scanned (#135252).

## Validation

Validation used the existing internal MDbg/xUnit debugger-test harness
with a private CoreRun and ProjectK/CoreCLR-only configuration:

- Targeted nested-byref invariant under target-only `GCStress=0x3`:
**1/1 passed**.
- Revised normal matrix: **35/35 passed**.
- Targeted stress matrix: **4/4 passed**.
- Windows x64 and x86 Checked native, CoreLib, and ReadyToRun builds
completed with zero warnings and errors.
- cDAC unit tests and generated usage/documentation checks passed for
the ExternalMemoryHandle descriptor changes.

Coverage includes moving GC, overlapping and nested byrefs, memory and
register homes, non-leaf registers, receiver mutation, constructors,
primitive/native-endian copy-back, nullable memory homes,
byref/byref-like/value-type results, collectible generics and layouts,
nested and exception-time eval, safe/rude/exception-unwind aborts, and
post-eval inspection. Runtime hashes and loaded modules were audited;
each abort scenario completed the callback, successfully evaluated
`Echo(42)`, detached, and exited MDbg with code 100. Test configurations
were restored and no debugger/test processes survived the runs.

These results were recorded on the validated pre-refresh candidate. The
final stack rebase preserved the layer's net diff; builds and tests were
not repeated after the history rewrite at the user's request.

The final native invocation cleanup layer is #133660.

---

<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>

Resolves #135245
Resolves #135247
Resolves #135249
Resolves #135251
Resolves #135252

> [!NOTE]
> This PR description and implementation were prepared with GitHub
Copilot. Commits that reuse am11's code retain the requested co-author
attribution.

---------

Co-authored-by: Adeel Mujahid <[email protected]>
Co-authored-by: Copilot App <[email protected]>