Use GCPROTECT_BEGININTERIOR to protect pointers into arrays during array marshalling#123688
Conversation
|
Tagging subscribers to this area: @dotnet/interop-contrib |
There was a problem hiding this comment.
Pull request overview
This PR fixes GC-related crashes in COM interop array marshalling by properly protecting interior pointers into managed arrays using GCPROTECT_BEGININTERIOR. The failures were introduced by PR #121362 which moved BSTR trailbyte tracking out of the sync block, causing transitions to managed code that could trigger GC during array marshalling.
Changes:
- Replaced manual pointer recalculation logic with
GCPROTECT_BEGININTERIORprotection - Updated 10 array marshalling functions to protect interior pointers during GC-triggering operations
- Removed obsolete "unprotected array" tracking code that manually checked if arrays moved
jkoritzinsky
force-pushed
the
bstr-marshaller-gcprotect
branch
from
62e5641 to
cdc3bcc
Compare
jkotas
left a comment
jkotas
left a comment
There was a problem hiding this comment.
My guess is that indexing from protected base would be a tiny bit cheaper than GC protected interior pointer (avoids stacks spills, multiplication is cheap). It does not really matter.
|
/ba-g nativeaot size failures that have been fixed in main |
Fixes #123271
Fixes #123269
Fixes #121690
Failures introduced by #121362 where we changed the string->BSTR array marshalling to call back into managed code. This introduces a transition where the code was not expecting one where GC can trigger.
Update the BSTR marshalling to use
GCPROTECT_BEGININTERIORto ensure that our pointer into the managed array is updated by the GC.A bunch of this code used various different patterns (manually adjusting offsets after a GC if one occurred, recalculating the offset into the array's data each iteration) instead of using
GCPROTECT_BEGININTERIOR. This PR adjusts all of the marshallers where GC could possibly occur (any sort of managed or possibly managed marshalling) to all useGCPROTECT_BEGININTERIOR.