Skip to content

[PROF-15376] Profiling: Disable live heap size profiling on Ruby 4 due to incompatibility#6022

Merged
ivoanjo merged 1 commit into
masterfrom
ivoanjo/disable-heap-live-size-ruby4
Jul 10, 2026
Merged

[PROF-15376] Profiling: Disable live heap size profiling on Ruby 4 due to incompatibility#6022
ivoanjo merged 1 commit into
masterfrom
ivoanjo/disable-heap-live-size-ruby4

Conversation

@ivoanjo

@ivoanjo ivoanjo commented Jul 10, 2026

Copy link
Copy Markdown
Member

What does this PR do?

The profiler supports two profile types related to heap profiling:

  • Heap Live Objects
  • Heap Live Size

This PR disables "Heap Live Size" on Ruby 4.0, leaving only "Heap Live Objects" available.

Motivation:

We've had reports of issues (such as #5936 when using heap live size on Ruby 4.0).

We've previously added a workaround in #5938 but we're now seeing issues with many different kinds of objects, not just classes, and thus we're disabling this feature on Ruby 4.0 until we can root cause the issue and fix it.

Change log entry

Yes. Profiling: Disable live heap size profiling on Ruby 4 due to incompatibility

Additional Notes:

This is how the message looks in practice:

I, [2026-07-10T11:24:56.368794 #2947286]  INFO -- datadog: [datadog]
Heap live size profiling is currently incompatible with Ruby 4 and has
been disabled. Heap live objects is not affected and remains enabled.

How to test the change?

I've included tests for this config change.

@ivoanjo
[PROF-15376] Profiling: Disable live heap size profiling on Ruby 4 du…
9d01d28 
…e to incompatibility

**What does this PR do?**

The profiler supports two profile types related to heap profiling:
* Heap Live Objects
* Heap Live Size

This PR disables "Heap Live Size" on Ruby 4.0, leaving only "Heap Live
Objects" available.

**Motivation:**

We've had reports of issues (such as
#5936 when using heap live
size on Ruby 4.0).

We've previously added a workaround in
#5938 but we're now
seeing issues with many different kinds of objects, not just classes,
and thus we're disabling this feature on Ruby 4.0 until we can root
cause the issue and fix it.

**Additional Notes:**

This is how the message looks in practice:

```
I, [2026-07-10T11:24:56.368794 #2947286]  INFO -- datadog: [datadog]
Heap live size profiling is currently incompatible with Ruby 4 and has
been disabled. Heap live objects is not affected and remains enabled.
```

**How to test the change?**

I've included tests for this config change.
@ivoanjo
ivoanjo requested review from a team as code owners July 10, 2026 11:30
@dd-octo-sts dd-octo-sts Bot added the profiling Involves Datadog profiling label Jul 10, 2026
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Jul 10, 2026 •
edited by datadog-official Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 0.00%
• Overall Coverage: 90.02% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9d01d28 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 10, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-10 11:55:57

Comparing candidate commit 9d01d28 in PR branch ivoanjo/disable-heap-live-size-ruby4 with baseline commit 3755d93 in branch master.

📊 Benchmarking dashboard

Found 2 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:profiling - Allocations ()

  • 🟩 throughput [+167080.618op/s; +212945.894op/s] or [+5.544%; +7.066%]

scenario:profiling - intern mixed existing and new

  • 🟩 throughput [+1.607op/s; +2.208op/s] or [+5.386%; +7.402%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:tracing - trace.to_digest - Continue

  • unstable throughput [-1836.082op/s; +1255.257op/s] or [-6.338%; +4.333%]

@ivoanjo
ivoanjo merged commit e81d4c0 into master Jul 10, 2026
588 checks passed
@ivoanjo
ivoanjo deleted the ivoanjo/disable-heap-live-size-ruby4 branch July 10, 2026 13:59
@dd-octo-sts dd-octo-sts Bot added this to the 2.38.0 milestone Jul 10, 2026
@ivoanjo ivoanjo added the identified-by:crashtracking Used to track internal stats for crashes! label Jul 13, 2026
@ivoanjo ivoanjo mentioned this pull request Jul 13, 2026
Merged
1 task
ivoanjo added a commit to DataDog/documentation that referenced this pull request Jul 13, 2026
@ivoanjo
[PROF-15376] Document that Ruby heap live size profiling is not compa…
2232b99 
…tible with Ruby 4 (#38117)

**What does this PR do?**

Update profiler docs to reflect the fact that heap live size profiling
is currently not compatible with Ruby 4, as per PROF-15376 JIRA and
DataDog/dd-trace-rb#6022 .

(Heap live objects profiling is unaffected.)

I did spot that vale doesn't like the word "currently" yet I believe
that's the most accurate representation of the state here -- we've had
to disable the feature due to bugs, but we do intent to restore it in
the future.
@navidemad navidemad mentioned this pull request Jul 16, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Strech Strech Strech approved these changes

Assignees

No one assigned

Labels

identified-by:crashtracking Used to track internal stats for crashes! profiling Involves Datadog profiling

Projects

None yet

Milestone

2.38.0

Development

Successfully merging this pull request may close these issues.

2 participants

@ivoanjo @Strech