Skip to content

[NO-TICKET] Fix rare profiler crash during shutdown in heap profiling cleanup#5920

Merged
ivoanjo merged 3 commits into
masterfrom
ivoanjo/fix-unintern-crash
Jun 19, 2026
Merged

[NO-TICKET] Fix rare profiler crash during shutdown in heap profiling cleanup#5920
ivoanjo merged 3 commits into
masterfrom
ivoanjo/fix-unintern-crash

Conversation

@ivoanjo

@ivoanjo ivoanjo commented Jun 18, 2026

Copy link
Copy Markdown
Member

What does this PR do?

This PR fixes a rare crash where the profiler can crash the Ruby process as the process itself is shutting down.

The native stack for the crash we observed looked like this:

rb_objspace_call_finalizer                       <- VM is shutting down
  rb_gc_obj_free
  stack_recorder_typed_data_free                 <- StackRecorder's free callback
    heap_recorder_free
    heap_record_free
      ... unintern_all_or_raise ...               <- libdatadog returned an error
      get_error_details_and_drop -> rb_str_new   <- allocates a Ruby String
        -> [BUG] object allocation during garbage collection phase

While freeing the heap recorder during teardown, we call into libdatadog to "unintern" the strings referenced by each tracked heap record. If that call returns an error, our error-handling path builds a Ruby String (to include the libdatadog error message) and then raises a Ruby exception.

But allocating objects and/or raising exceptions in this situation is not correct, and causes the crash.

The fix is to skip the uninterning entirely during teardown: the whole managed string storage is dropped immediately after heap_recorder_free returns (see stack_recorder_typed_data_free), so there's no point adjusting individual string refcounts -- and, crucially, it means we no longer allocate or raise from inside the GC free callback. Regular operation, where uninterning and raising on error is both correct and safe, is unchanged.

Motivation:

Fix rare crash.

Change log entry

Yes. Fix rare profiler crash during shutdown in heap profiling cleanup

Additional Notes:

It's not clear why libdatadog failed to unintern, but the fix is needed regardless: the error-handling path (allocate + raise) is not safe to run from a GC free callback.

How to test the change?

Simulating failures on specific libdatadog APIs in the middle of C code is really awkward so I'm relying only on the existing test coverage for this one.

… cleanup

**What does this PR do?**

This PR fixes a rare crash where the profiler can crash the Ruby
process as the process itself is shutting down.

The native stack for the crash we observed looked like this:

```
rb_objspace_call_finalizer                       <- VM is shutting down
  rb_gc_obj_free
  stack_recorder_typed_data_free                 <- StackRecorder's free callback
    heap_recorder_free
    heap_record_free
      ... unintern_all_or_raise ...               <- libdatadog returned an error
      get_error_details_and_drop -> rb_str_new   <- allocates a Ruby String
        -> [BUG] object allocation during garbage collection phase
```

While freeing the heap recorder during teardown, we call into libdatadog
to "unintern" the strings referenced by each tracked heap record. If that
call returns an error, our error-handling path builds a Ruby String (to
include the libdatadog error message) and then raises a Ruby exception.

But allocating objects and/or raising exceptions in this situation is not
correct, and causes the crash.

The fix is to skip the uninterning entirely during teardown: the whole managed
string storage is dropped immediately after `heap_recorder_free` returns
(see `stack_recorder_typed_data_free`), so there's no point adjusting
individual string refcounts -- and, crucially, it means we no longer
allocate or raise from inside the GC free callback. Regular operation,
where uninterning and raising on error is both correct and safe, is
unchanged.

**Motivation:**

Fix rare crash.

**Additional Notes:**

It's not clear why libdatadog failed to unintern, but the
fix is needed regardless: the error-handling path (allocate + raise) is
not safe to run from a GC free callback.

**How to test the change?**

Simulating failures on specific libdatadog APIs in the middle of C code
is really awkward so I'm relying only on the existing test coverage for
this one.
@ivoanjo
ivoanjo requested review from a team as code owners June 18, 2026 17:21
@dd-octo-sts dd-octo-sts Bot added the profiling Involves Datadog profiling label Jun 18, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 18, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 89.99% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e4f1535 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jun 18, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-19 10:24:25

Comparing candidate commit e4f1535 in PR branch ivoanjo/fix-unintern-crash with baseline commit bff118c in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

@ivoanjo
ivoanjo requested a review from eregon June 19, 2026 07:48
@ivoanjo
ivoanjo merged commit 2980cb2 into master Jun 19, 2026
588 checks passed
@ivoanjo
ivoanjo deleted the ivoanjo/fix-unintern-crash branch June 19, 2026 12:07
@dd-octo-sts dd-octo-sts Bot added this to the 2.36.0 milestone Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

profiling Involves Datadog profiling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants