dotnet/runtime#130698
reported a Linux/x64 regression in Memory<T>.Pin(Size: 512). The regression
was bisected to commit
8f365c934088,
which changed GC handle publication from a plain pointer store to
VolatileStore in PR #130213.
The correctness change is required: publishing a GC handle needs release semantics. On x64, however, a release store needs no additional hardware instruction. The regression came from Clang retaining an unnecessary branch around the compiler barrier:
call HndWriteBarrierWorker
jmp store # target is the next emitted instruction
store:
mov %rbx, (%r14)The adopted fix replaces the Clang definition of VOLATILE_MEMORY_BARRIER()
itself:
#if defined(__clang__)
#define VOLATILE_MEMORY_BARRIER() __atomic_signal_fence(__ATOMIC_SEQ_CST)
#else
#define VOLATILE_MEMORY_BARRIER() asm volatile ("" : : : "memory")
#endifThis expresses the required compiler barrier without an opaque inline assembly
memory clobber or an LLVM atomic store. It emits no fence instruction on x64
and restores direct fallthrough into the pointer store. See
Final Decision for why the macro-wide form was chosen over
a store-specific RELEASE specialization.
VolatileStore promises:
- The store is not optimized away.
- Operations before the store in program order are not moved after it.
- Atomicity is not generally guaranteed, although aligned scalar stores are atomic on the platforms relevant here.
The second property is the release portion of the contract.
x86 and x64 hardware already provide sufficient ordering for this use. The compiler must still be prevented from moving earlier memory operations after the store.
For MSVC on x86/x64, VOLATILE_MEMORY_BARRIER() is empty. MSVC defaults to
Microsoft volatile semantics on these targets, under which a volatile write
has release semantics and a volatile read has acquire semantics. The volatile
access itself therefore supplies the compiler ordering.
See Microsoft's volatile documentation.
GCC and Clang use ISO-style volatile semantics. A volatile access must occur,
but it does not by itself constrain ordinary non-volatile memory operations in
the way required by VolatileStore.
The previous GNU x64 implementation supplied the missing compiler ordering with:
#define VOLATILE_MEMORY_BARRIER() asm volatile ("" : : : "memory")This emits no machine bytes, but it is not a no-op to the optimizer. It is an opaque side-effecting inline assembly operation with a memory clobber.
Defining the macro to nothing for Clang/x64 would remove the branch, but it
would also violate the documented VolatileStore and VolatileLoad
contracts. Strong x64 hardware ordering does not prevent compiler reordering.
The local reproduction used:
- Runtime checkout commit:
2177d80507a - Configuration: Linux x64 Release with native PGO
- Official build container:
mcr.microsoft.com/dotnet-buildtools/prereqs@sha256:4b0004c00cba047397f3a03af56a6a196b8002d71af1852c2508386b4b3f090c - Compiler: Clang 22.1.5
- Linker: LLD 22.1.5
- PGO data:
.nuget/packages/optimization.linux-x64.pgo.coreclr/1.0.0-prerelease.26318.1/data/coreclr.profdata
The exact object containing the reproduced function was:
artifacts/obj/coreclr/linux.x64.Release/vm/wks/
CMakeFiles/cee_wks_core.dir/__/__/gc/gchandletable.cpp.o
The issue is present in that object, before linking, so it is compiler codegen rather than a linker transformation.
Before GC handle publication used VolatileStore, Clang emitted:
test %rbx, %rbx
je store
mov %r14, %rdi
mov %rbx, %rsi
call HndWriteBarrierWorker
store:
mov %rbx, (%r14)With the generic GNU VolatileStore, it emitted:
test %rbx, %rbx
je store
mov %r14, %rdi
mov %rbx, %rsi
call HndWriteBarrierWorker
jmp store
store:
mov %rbx, (%r14)The jump target is the next emitted instruction. Although the empty inline assembly has no bytes, it remains an opaque operation while LLVM constructs and optimizes the control-flow graph.
An x64 release atomic store also uses a plain mov. The unwanted branch is
not a hardware requirement of atomicity or release ordering.
The generic __atomic_store builtin changes the LLVM IR shape. For the pointer
stored by HndAssignHandle, optimized Clang 22 IR contains:
nonnull:
%valueAsInt = ptrtoint ptr %value to i64
br label %store
store:
%storedValue = phi i64 [ %valueAsInt, %nonnull ], [ 0, %null ]
store atomic volatile i64 %storedValue, ptr %handle releaseThe pointer is converted to its integer object representation. Null is folded
to integer zero, producing a phi with different incoming definitions.
Codegen then materializes zero on the null path:
call HndWriteBarrierWorker
jmp store
null:
xorl %r14d, %r14d
store:
movq %r14, (%rbx)At this point the jump is necessary to skip the null-path xor, even though a
better lowering could have retained the original pointer register on both
paths.
Atomic operations also retain their abstract optimizer restrictions even when
the target instruction is a normal mov. LLVM currently marks the machine
memory operand for every atomic operation as volatile and handles it
conservatively. See the
LLVM Atomic Instructions and Concurrency Guide.
__atomic_store_n is not a general replacement for this template. For
example, instantiating VolatileStore<double> fails because _n requires an
integer or pointer operand. The generic __atomic_store accepts arbitrary
object types, but introduces the legalization behavior above.
The store-specific signal-fence implementation produces:
fence syncscope("singlethread") release
store volatile ptr %value, ptr %handleImportant differences from __atomic_store are:
- The pointer remains a pointer SSA value.
- No integer
phior null-path materialization is introduced. - The store is the existing non-atomic volatile store required by the
VolatileStorecontract. - The fence represents compiler ordering separately from the memory access.
Clang lowers the signal fence to a zero-sized compiler barrier pseudo on x64. The final code is:
testq %rbx, %rbx
je store
movq %r14, %rdi
movq %rbx, %rsi
callq HndWriteBarrierWorker
store:
movq %rbx, (%r14)There is no hardware fence and no unconditional branch.
__atomic_signal_fence does not turn ordinary C++ accesses into formally
atomic inter-thread accesses. That is not a regression from the existing
VolatileStore contract, which explicitly does not promise general
atomicity. It provides the compiler ordering needed by this runtime primitive,
while x64 supplies the required hardware ordering.
Another working implementation is to replace the Clang/AMD64 definition of
VOLATILE_MEMORY_BARRIER() with:
#define VOLATILE_MEMORY_BARRIER() \
__atomic_signal_fence(__ATOMIC_SEQ_CST)SEQ_CST is appropriate for a shared macro because the macro is used:
- after volatile loads, where acquire behavior is needed;
- before volatile stores, where release behavior is needed; and
- as a general full compiler barrier.
This is the closest modeled replacement for the old full inline-assembly memory clobber. Both emit no instructions on x64. The signal fence is more optimization-friendly because LLVM understands its ordering semantics instead of treating it as arbitrary inline assembly.
The macro-wide change has a broader behavioral and code-layout surface. The
store-specific RELEASE specialization is the tighter fix for this regression
and states the exact VolatileStore requirement.
| Implementation | Result |
|---|---|
| Plain store before the correctness fix | No jump, but no compiler release barrier |
| Empty inline asm plus volatile store | Reproduced useless jump |
Generic __atomic_store(..., RELEASE) |
Plain x64 mov, but introduced integer phi, null materialization, and a branch |
__atomic_store_n(..., RELEASE) |
Not valid for all VolatileStore<T> instantiations, including double |
| Duplicate stores at the GC handle call site | Clang tail-merged the stores and recreated the branch |
| Conditional implementation at the call site | Could improve codegen, but leaked compiler/architecture policy outside volatile.h |
Macro-wide signal_fence(SEQ_CST) |
No jump; broader change |
Store-specific signal_fence(RELEASE) |
No jump; narrowest correct workaround |
A focused benchmark reproduced the issue's
Memory<byte>.Pin(Size: 512) operation. Each process used:
- A 512-byte
Memory<byte> - 2,000,000 warmup operations
- 21 measured samples
- 5,000,000 pin/dispose operations per sample
- CPU affinity fixed to CPU 2
DOTNET_TieredCompilation=0
Tiering was disabled because short local runs with tiered PGO showed tier transition clusters. The native GC handle operation under investigation is unchanged by that setting.
Each runtime root used matching CoreLib and JIT artifacts. Comparison roots
differed in libcoreclr.so.
Four fresh processes per version produced:
| Version | Process medians (ns/op) | Aggregate median |
|---|---|---|
| Original inline-asm barrier | 14.967, 14.988, 15.016, 15.070 | 15.002 |
Store-specific signal_fence(RELEASE) |
14.223, 14.249, 14.384, 14.447 | 14.317 |
The current fix recovered 0.685 ns/op. The original inline-assembly version
was 4.8% slower than the current version.
In a contemporaneous three-way comparison:
| Version | Aggregate median (ns/op) |
|---|---|
| Original inline-asm barrier | 15.062 |
Macro-wide signal_fence(SEQ_CST) |
13.901 |
Store-specific signal_fence(RELEASE) |
14.456 |
A separate direct comparison measured:
| Version | Aggregate median (ns/op) |
|---|---|
Macro-wide signal_fence(SEQ_CST) |
13.947 |
Store-specific signal_fence(RELEASE) |
14.331 |
The macro-wide build was faster in these measurements. This does not show
that SEQ_CST is intrinsically cheaper than RELEASE: both fences emit no
instruction, and StoreObjectInHandle is instruction-for-instruction
identical in the two fixed binaries.
The macro-wide build changes every Clang/x64 use of
VOLATILE_MEMORY_BARRIER() in libcoreclr, and it changes native code layout.
The benchmark therefore cannot isolate the ordering constant. A strict
RELEASE versus SEQ_CST comparison would require two store-specific builds
whose only source difference is the fence ordering.
The final store-specific release-fence implementation was validated by:
- Rebuilding the exact native-PGO
gchandletable.cpp.o. - Inspecting the object disassembly and confirming direct fallthrough.
- Inspecting the linked
libcoreclr.soand confirming the same code. - Completing the full CoreCLR
ninja installtarget with Clang 22. - Running
git diff --check.
Example object inspection:
llvm-objdump-21 -d --no-show-raw-insn \
--disassemble-symbols='_ZN15GCHandleManager19StoreObjectInHandleEP14OBJECTHANDLE__P6Object' \
artifacts/obj/coreclr/linux.x64.Release/vm/wks/\
CMakeFiles/cee_wks_core.dir/__/__/gc/gchandletable.cpp.oExample benchmark invocation:
DOTNET_EnableDiagnostics=0 \
DOTNET_TieredCompilation=0 \
taskset -c 2 \
/tmp/pinbench/dotnet-release/dotnet \
/tmp/pinbench/app/PinBench.dllThe adopted fix is the macro-wide replacement: for Clang, define
VOLATILE_MEMORY_BARRIER() as __atomic_signal_fence(__ATOMIC_SEQ_CST) in
the x86/AMD64 GNU branch of both src/coreclr/inc/volatile.h and its
standalone-GC copy src/coreclr/gc/env/volatile.h (the two share an include
guard and must stay in sync). The store-specific VolatileStore
specialization was removed.
Reasons the macro-wide form is correct and preferable to the store-specific
RELEASE specialization:
VOLATILE_MEMORY_BARRIER()is used as a full compiler barrier outside this header (vm/object.cpp,gc/allocation.cpp), and after volatile loads where acquire ordering is needed.SEQ_CSTis the exact modeled equivalent of the old full memory clobber — nothing is weakened anywhere.- It fixes
VolatileLoadand every other macro user too, not justVolatileStore; the inline-asm branch hazard existed on all of them. - Policy stays in one place instead of forking
VolatileStoreper compiler. - It benchmarked fastest of all variants (13.9 ns/op vs 14.3 store-specific vs 15.0 original).
GCC keeps the battle-tested asm clobber: the regression is Clang-specific, though GCC 14 was verified to honor the signal fence identically (see below).
Beyond the original validation steps, the final form was verified by:
- A discriminating dead-store-elimination test: with no barrier, Clang 21
and GCC 14 at
-O2eliminate a globally-observable store preceding the volatile publication store; with__atomic_signal_fence(SEQ_CSTorRELEASE) the store is preserved, byte-identical to the inline-asm clobber version. This confirms the fence is a real compiler barrier for non-atomic accesses, not just for atomics. - Rebuilding the exact native-PGO
gchandletable.cpp.o(wks flavor, including its precompiled header) in the official build container and confirmingStoreObjectInHandlefalls directly through into the pointer store with no unconditional branch.