[NO-TICKET] Fix rare profiler crash during shutdown in heap profiling cleanup#5920
Conversation
… cleanup
**What does this PR do?**
This PR fixes a rare crash where the profiler can crash the Ruby
process as the process itself is shutting down.
The native stack for the crash we observed looked like this:
```
rb_objspace_call_finalizer <- VM is shutting down
rb_gc_obj_free
stack_recorder_typed_data_free <- StackRecorder's free callback
heap_recorder_free
heap_record_free
... unintern_all_or_raise ... <- libdatadog returned an error
get_error_details_and_drop -> rb_str_new <- allocates a Ruby String
-> [BUG] object allocation during garbage collection phase
```
While freeing the heap recorder during teardown, we call into libdatadog
to "unintern" the strings referenced by each tracked heap record. If that
call returns an error, our error-handling path builds a Ruby String (to
include the libdatadog error message) and then raises a Ruby exception.
But allocating objects and/or raising exceptions in this situation is not
correct, and causes the crash.
The fix is to skip the uninterning entirely during teardown: the whole managed
string storage is dropped immediately after `heap_recorder_free` returns
(see `stack_recorder_typed_data_free`), so there's no point adjusting
individual string refcounts -- and, crucially, it means we no longer
allocate or raise from inside the GC free callback. Regular operation,
where uninterning and raising on error is both correct and safe, is
unchanged.
**Motivation:**
Fix rare crash.
**Additional Notes:**
It's not clear why libdatadog failed to unintern, but the
fix is needed regardless: the error-handling path (allocate + raise) is
not safe to run from a GC free callback.
**How to test the change?**
Simulating failures on specific libdatadog APIs in the middle of C code
is really awkward so I'm relying only on the existing test coverage for
this one.
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: e4f1535 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-06-19 10:24:25 Comparing candidate commit e4f1535 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 1 unstable metrics.
|
What does this PR do?
This PR fixes a rare crash where the profiler can crash the Ruby process as the process itself is shutting down.
The native stack for the crash we observed looked like this:
While freeing the heap recorder during teardown, we call into libdatadog to "unintern" the strings referenced by each tracked heap record. If that call returns an error, our error-handling path builds a Ruby String (to include the libdatadog error message) and then raises a Ruby exception.
But allocating objects and/or raising exceptions in this situation is not correct, and causes the crash.
The fix is to skip the uninterning entirely during teardown: the whole managed string storage is dropped immediately after
heap_recorder_freereturns (seestack_recorder_typed_data_free), so there's no point adjusting individual string refcounts -- and, crucially, it means we no longer allocate or raise from inside the GC free callback. Regular operation, where uninterning and raising on error is both correct and safe, is unchanged.Motivation:
Fix rare crash.
Change log entry
Yes. Fix rare profiler crash during shutdown in heap profiling cleanup
Additional Notes:
It's not clear why libdatadog failed to unintern, but the fix is needed regardless: the error-handling path (allocate + raise) is not safe to run from a GC free callback.
How to test the change?
Simulating failures on specific libdatadog APIs in the middle of C code is really awkward so I'm relying only on the existing test coverage for this one.