Profiling: Skip class/module memsize on Ruby 4 to avoid heap profiling SIGSEGV#5938
Conversation
…g SIGSEGV On Ruby 4.0, computing the memsize of a class/module/iclass walks the per-namespace class extensions (rb_class_classext_foreach -> classext_memsize -> rb_id_table_memsize) and can crash the VM with a SIGSEGV when called on objects resurrected via ObjectSpace._id2ref during heap profiling. Add T_CLASS/T_MODULE/T_ICLASS to the existing crash-path denylist in ruby_obj_memsize_of, gated behind a new NO_SAFE_CLASS_MEMSIZE define for Ruby >= 4. On older Rubies the preprocessed code is unchanged. Class objects contribute negligibly to a heap size profile. See DataDog#5936
ivoanjo
left a comment
ivoanjo
left a comment
There was a problem hiding this comment.
Thanks for submitting this! I think it makes sense to merge this in -- I've just submitted a few suggested tweaks to the comments :D
In particular, at this moment I believe this is an underlying bug on the Ruby VM -- just one that hasn't been spotted/fixed yet since for most Ruby apps rb_obj_memsize_of/ObjectSpace.memsize_of is very rarely called, whereas dd-trace-rb calls it all the time, so we're in a way "fuzzing" the Ruby VM and finding the one-in-a-million needle.
It's also not clear to me that the issue is related to the _id2ref -- we did have some issues regarding it on Ruby 4 earlier on, but the current approach we're using is expected to be safe and supported (although we have plans to replace it), so I think that may be a bit of a "red herring" in this situation.
Other than that, I think the workaround makes sense until we can figure out a way of reproducing this and then report it upstream.
Co-authored-by: Ivo Anjo <[email protected]>
Co-authored-by: Ivo Anjo <[email protected]>
Co-authored-by: Ivo Anjo <[email protected]>
ivoanjo
left a comment
ivoanjo
left a comment
There was a problem hiding this comment.
👍 Looks great, thank you!
I'll take care of getting this merged :D
|
…e to incompatibility **What does this PR do?** The profiler supports two profile types related to heap profiling: * Heap Live Objects * Heap Live Size This PR disables "Heap Live Size" on Ruby 4.0, leaving only "Heap Live Objects" available. **Motivation:** We've had reports of issues (such as DataDog#5936 when using heap live size on Ruby 4.0). We've previously added a workaround in DataDog#5938 but we're now seeing issues with many different kinds of objects, not just classes, and thus we're disabling this feature on Ruby 4.0 until we can root cause the issue and fix it. **Additional Notes:** This is how the message looks in practice: ``` I, [2026-07-10T11:24:56.368794 #2947286] INFO -- datadog: [datadog] Heap live size profiling is currently incompatible with Ruby 4 and has been disabled. Heap live objects is not affected and remains enabled. ``` **How to test the change?** I've included tests for this config change.
3 participants
What does this PR do?
Skips computing
rb_obj_memsize_offorT_CLASS/T_MODULE/T_ICLASSobjects on Ruby 4.0+, where it can crash the VM with a SIGSEGV during heap profiling.ruby_obj_memsize_ofalready keeps a denylist ofrb_obj_memsize_ofpaths that crash the VM (e.g.T_NODE). This adds the class-like types to that denylist on Ruby 4.0+, gated by a newNO_SAFE_CLASS_MEMSIZEcompile-time define. On older Rubies the define is absent, so the preprocessed code is byte-for-byte unchanged.Motivation:
With experimental heap profiling enabled on Ruby 4.0, we saw recurring SIGSEGV crashes on production Sidekiq workers (#5936). During heap profile serialization, the recorder resurrects each tracked object via
ObjectSpace._id2refand callsrb_obj_memsize_ofon it. For class objects,rb_obj_memsize_ofwalks Ruby 4.0's per-namespace class extensions (rb_class_classext_foreach→classext_memsize→rb_id_table_memsize) and dereferences invalid classext memory, killing the process. Class objects contribute negligibly to a heap size profile, so skipping them removes the crash with no meaningful loss of signal. The full root-cause analysis is in #5936.Change log entry
Yes. Profiling: Fix a SIGSEGV crash that could happen with experimental heap profiling enabled on Ruby 4.0.
Additional Notes:
The gate is a new
NO_SAFE_CLASS_MEMSIZEdefine, set forRUBY_VERSION >= "4", following the existing extconf.rb convention for Ruby-4-specific behavior. A complete fix would also need Ruby-side hardening ofclassext_memsize/rb_class_classext_foreach, but skipping class memsize is enough to stop the crash.How to test the change?
Added a regression test in
stack_recorder_spec.rb: it tracks aClassas a live heap object and checks that the profiler reports it without crashing, with a heap-live-size of 0 on Ruby 4.0+ (and the realObjectSpace.memsize_ofon older Rubies). Verified on Linux + Ruby 4.0 (tracer-4.0): the fullstack_recorder_specpasses (64 examples, 0 failures). On Ruby < 4 the change is a no-op at the preprocessor level.