Skip to content

[PROF-6071] Detect when libdatadog version does not match expected version#5484

Merged
ivoanjo merged 7 commits into
masterfrom
ivoanjo/prof-6071-detect-libdatadog-changes
Mar 23, 2026
Merged

[PROF-6071] Detect when libdatadog version does not match expected version#5484
ivoanjo merged 7 commits into
masterfrom
ivoanjo/prof-6071-detect-libdatadog-changes

Conversation

@ivoanjo

@ivoanjo ivoanjo commented Mar 20, 2026

Copy link
Copy Markdown
Member

What does this PR do?

This PR adds a runtime check to detect when the libdatadog gem version doesn't match the version that the native extensions were compiled against.

When the datadog gem's native extensions are built, we now store the libdatadog version as a C preprocessor define (EXPECTED_LIBDATADOG_VERSION).
At runtime, during extension initialization, we compare this against Libdatadog::VERSION and raise a helpful error if they don't match.

Motivation:

Fixes #2242.

This is a rare but confusing issue that can happen when there are multiple libdatadog point releases that are compatible with the same datadog gem version. Historically this happened with ddtrace (the predecessor to the datadog gem) -- e.g. ddtrace 1.3.0 could use either libdatadog 0.7.0.1.0 or 0.7.0.1.1.

In practice, no released version of the datadog gem has ever been affected by this issue, as each version pins to a specific libdatadog version with no other compatible releases. But this remains a useful defensive measure in case point releases are ever made in the future.

Change log entry

Yes. Core: Detect when libdatadog version does not match expected version.

Additional Notes:

I chose to use rb_eval_string to do the check since doing it in C is a lot more boilerplate and anyway this only happens once when an extension is loaded.

How to test the change?

Change includes test coverage.

I've also tested this locally by building a modern libdatadog with a dummy version:

$ bundle info libdatadog
  * libdatadog (29.0.0.1.0)
	Summary: Library of common code used by Datadog Continuous Profiler for Ruby
	Homepage: https://docs.datadoghq.com/tracing/
	Source Code: https://github.com/datadog/libdatadog-rb
	Path: .rvm/gems/ruby-3.2.9/gems/libdatadog-29.0.0.1.0-x86_64-linux
	Reverse Dependencies: 
		datadog (2.30.0) depends on libdatadog (~> 29.0.0.1.0)
$ bundle exec rake clean compile
# ...
$ bundle add libdatadog --version 29.0.0.1.1234
# ...
$ bundle info libdatadog
  * libdatadog (29.0.0.1.1234)
	Summary: Library of common code used by Datadog Continuous Profiler for Ruby
	Homepage: https://docs.datadoghq.com/tracing/
	Source Code: https://github.com/datadog/libdatadog-rb
	Path: .rvm/gems/ruby-3.2.9/gems/libdatadog-29.0.0.1.1234-x86_64-linux
	Reverse Dependencies: 
		datadog (2.30.0) depends on libdatadog (~> 29.0.0.1.0)
$ DD_PROFILING_ENABLED=true bundle exec ddprofrb exec ruby -e "sleep 1"
W, [2026-03-20T11:09:27.758853 #162429]  WARN -- datadog: [datadog] Profiling was requested
 but is not supported, profiling disabled: There was an error loading the profiling native
 extension due to 'LoadError Failed to load the profiling loader extension. To fix this,
 please remove and then reinstall datadog (Details: The `datadog` gem needs to be reinstalled
 whenever the `libdatadog` gem version is changed. The currently-installed version of
 `datadog` was built to work with `libdatadog` gem version 29.0.0.1.0 but the 
currently-loaded version of `libdatadog` is 29.0.0.1.1234. To fix this, reinstall the
 `datadog` gem (e.g. `bundle exec gem pristine datadog`) or contact Datadog support for help
 at <https://docs.datadoghq.com/help/>.)' at
 lib/datadog/profiling/load_native_extension.rb:6:in `rescue in <top (required)>''
I, [2026-03-20T11:09:27.759736 #162429]  INFO -- datadog: [datadog] DATADOG CONFIGURATION -
 CORE - {"date":"2026-03-20T11:09:27Z","os_name":"x86_64-pc-
linux","version":"2.30.0","lang":"ruby","lang_version":"3.2.9","env":null,"service":"-
e","dd_version":null,"debug":false,"tags":null,"runtime_metrics_enabled":false,"vm":"ruby-
3.2.9","health_metrics_enabled":false,"profiling_enabled":false,"dynamic_instrumentation_enabled":false}

ivoanjo and others added 3 commits March 20, 2026 10:57
@ivoanjo @claude
[PROF-6071] Detect when libdatadog version does not match expected ve…
dbe3c82 
…rsion

**What does this PR do?**

This PR adds a runtime check to detect when the `libdatadog` gem version
doesn't match the version that the native extensions were compiled against.

When the `datadog` gem's native extensions are built, we now store the
`libdatadog` version as a C preprocessor define (`EXPECTED_LIBDATADOG_VERSION`).
At runtime, during extension initialization, we compare this against
`Libdatadog::VERSION` and raise a helpful error if they don't match.

**Motivation:**

Fixes <#2242>.

This is a rare but confusing issue that can happen when there are multiple
`libdatadog` point releases that are compatible with the same `datadog` gem
version. Historically this happened with `ddtrace` (the predecessor to the
`datadog` gem) -- e.g. ddtrace 1.3.0 could use either libdatadog 0.7.0.1.0
or 0.7.0.1.1.

In practice, no released version of the `datadog` gem has ever been affected
by this issue, as each version pins to a specific libdatadog version with no
other compatible releases. But this remains a useful defensive measure in case
point releases are ever made in the future.

**Change log entry**

Yes. Core: Detect when `libdatadog` version does not match expected version.

**Additional Notes:**

I chose to use `rb_eval_string` to do the check since doing it in C is a lot
more boilerplate and anyway this only happens once when an extension is loaded.

**How to test the change?**

Change includes test coverage.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@ivoanjo
Minor: Fix leftover reference to old loader mechanism
556a6e8
@ivoanjo
Use LoadError when emitting issue
c966b7a 
Our existing begin/rescue code expects `LoadError`s.
@ivoanjo
ivoanjo requested review from a team as code owners March 20, 2026 11:15
@github-actions github-actions Bot added the profiling Involves Datadog profiling label Mar 20, 2026
@pr-commenter

pr-commenter Bot commented Mar 20, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-03-23 09:37:20

Comparing candidate commit 99db6f3 in PR branch ivoanjo/prof-6071-detect-libdatadog-changes with baseline commit 78f9609 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Strech
Strech approved these changes Mar 20, 2026
View reviewed changes

@Strech Strech left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼

Comment thread ext/libdatadog_extconf_helpers.rb Outdated
@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Mar 20, 2026 •
edited
Loading

Copy link
Copy Markdown

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 95.65%
• Overall Coverage: 95.13% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 99db6f3 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

ivoanjo and others added 3 commits March 20, 2026 14:58
@ivoanjo @Strech
Update ext/libdatadog_extconf_helpers.rb
69e69b5 
Co-authored-by: Sergey Fedorov <[email protected]>
@ivoanjo
Minor: Make linter happy
ff11894
@ivoanjo
Make sure quotation of number is part of define
99db6f3 
The quotation here needs to be part of the string since that's going
into the `Makefile`.

I missed this was subtly broken when we tried to avoid
too-much-escaping.
@ivoanjo
ivoanjo merged commit dbb8f0f into master Mar 23, 2026
631 checks passed
@ivoanjo
ivoanjo deleted the ivoanjo/prof-6071-detect-libdatadog-changes branch March 23, 2026 10:34
@github-actions github-actions Bot added this to the 2.31.0 milestone Mar 23, 2026
@y9v y9v mentioned this pull request Apr 16, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Strech Strech Strech approved these changes

Assignees

No one assigned

Labels

profiling Involves Datadog profiling

Projects

None yet

Milestone

2.31.0

Development

Successfully merging this pull request may close these issues.

Profiling native extension does not detect libdatadog upgrades/downgrades

2 participants

@ivoanjo @Strech