DI: support per-method iseqs for line probes on pre-loaded files by p-datadog · Pull Request #5501 · DataDog/dd-trace-rb

p-datadog · 2026-03-24T23:38:43Z

What does this PR do?
When a file's whole-file (:top) iseq has been garbage collected, the backfill now stores surviving per-method iseqs and the instrumenter can use them to target line probes.

Motivation:
86% of files loaded before DI starts have their whole-file iseq GC'd but still have surviving per-method iseqs (measured on a Rails app with 146 gems: 1987 of 2309 missing files had 22739 surviving method iseqs). Previously these files were completely untargetable by line probes.

Change log entry
Yes. Live Debugger line probes can now target third-party and Ruby standard library code loaded before datadog gem is loaded.

Additional Notes:

Depends on DI: backfill CodeTracker registry with iseqs for pre-loaded files #5496 (backfill) which depends on DI: add a C extension #5111 (C extension)
backfill_registry now stores per-method iseqs in a separate per_method_registry keyed by path
New iseq_for_line(suffix, line) method checks whole-file iseq first, falls back to per-method iseqs by checking trace_points for the target line
Instrumenter uses iseq_for_line when available, falls back to iseqs_for_path_suffix for compatibility with older code trackers
14% of missing files (322) have no surviving iseqs at all — these are files that only run setup code without defining methods

How to test the change?
Unit tests added + manual testing via gobo targeting stdlib with partial iseqs:

The screenshot shows Set#add captured with argument 1. The gobo demo calls Set.new([1, 2, 3]) then s.add(4) and s.add(2), but Set.new internally calls add for each element (1, 2, 3), so the probe fires on add(1) first. Due to the DI rate limit (one snapshot per second by default), only the first invocation is captured.

pr-commenter · 2026-03-25T00:03:18Z

Benchmarks

Benchmark execution time: 2026-04-25 22:48:20

Comparing candidate commit 24f87ca in PR branch di-per-method-iseq with baseline commit bab5284 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

github-actions · 2026-03-27T23:45:49Z

Typing analysis

Note: Ignored files are excluded from the next sections.

`steep:ignore` comments

This PR introduces 2 steep:ignore comments, and clears 1 steep:ignore comment.

steep:ignore comments (+2-1)

❌ Introduced:

lib/datadog/di/instrumenter.rb:341
lib/datadog/di/instrumenter.rb:343

✅ Cleared:

lib/datadog/di/instrumenter.rb:340

Untyped methods

This PR introduces 1 partially typed method, and clears 1 partially typed method. It increases the percentage of typed methods from 61.33% to 61.37% (+0.04%).

Partially typed methods (+1-1)

❌ Introduced:

sig/datadog/di/code_tracker.rbs:16
└── def iseqs_for_path_suffix: (String suffix) -> untyped

✅ Cleared:

sig/datadog/di/code_tracker.rbs:15
└── def iseqs_for_path_suffix: (String suffix) -> untyped

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

* origin/di-per-method-iseq: (23 commits) Improve DITargetNotInRegistry error messages Update remote config test for new error message format Fix throwable integration test to include stacktrace Add integration test for line probe via per-method iseq Support per-method iseqs for line probes on pre-loaded files Fix Steep: allow nil for @current_components Fix StandardRB: add parens to ternary, remove extra blank line Fix backfill_registry test failures Disable GC during backfill integration test to prevent iseq collection Fix backfill_registry tests on Ruby < 3.1 (iseq_type unavailable) Initialize @current_components to suppress Ruby 2.6/2.7 warning Return nil explicitly from backfill_registry Remove respond_to?(:all_iseqs) guard from backfill_registry Add tests for calling backfill_registry twice Fix inaccurate comment: first_lineno == 0 heuristic matches iseq_type Document iseq_type Ruby 3.1 dependency and two-strategy backfill Review fixes: doc comments, error handling test coverage, spec_helper require Guard rb_iseq_type behind have_func for Ruby < 3.1 compat Add DI.iseq_type C extension; use type instead of first_lineno in backfill Stub backfill_registry in pre-existing tests ...

datadog-prod-us1-6 · 2026-03-28T06:02:02Z

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 97.58%
• Overall Coverage: 97.20% (-0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 24f87ca | Docs | Datadog PR Page | Give us feedback!}

p-datadog · 2026-04-08T21:19:06Z

@codex review

The backfill_registry docstring said "per-method iseqs require instrumenter changes and will be supported in a follow-up." This PR is that follow-up — the comment is now stale. Co-Authored-By: Claude <[email protected]>

…ent section The section said "Code loaded before the tracer initializes cannot be instrumented with line probes." After #5496 (whole-file iseq backfill) and this PR (per-method iseq fallback), that limitation no longer exists. Remove the section entirely rather than rewrite it. Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>

Address review comment: compile_file-produced :top iseqs with first_lineno != 0 fell through the whole_file check into per_method_registry. Probes targeting these iseqs fail because targeted TracePoints are bound to the specific iseq object. - Fixed in lib/datadog/di/code_tracker.rb:90 (type guard in else branch) - Added test in spec/datadog/di/code_tracker_spec.rb Co-Authored-By: Claude <[email protected]>

Address review comment: iseq_for_line matched any trace point event at the target line, including :call. Since the instrumenter subscribes to [:line, :return, :b_return], a :call-only line would cause TracePoint#enable to raise. Now filters to executable event types so the probe goes to pending state instead of erroring. - Fixed in lib/datadog/di/code_tracker.rb:275 (event type filter) - Added tests in spec/datadog/di/code_tracker_spec.rb Co-Authored-By: Claude <[email protected]>

The second entry (with jruby) overwrote the first. Keep only the entry without jruby, matching the reviewer's suggestion and the format of adjacent entries. Co-Authored-By: Claude <[email protected]>

- Comment 1 (compile_file guard): replace MRI jargon and "known limitation" with concrete user impact (probe silently produces no snapshots) and practical likelihood on Ruby < 3.1 - Comment 2 (event filter): clarify that the `def` line example refers to the defined method's own iseq, not the enclosing scope Co-Authored-By: Claude <[email protected]>

State the consequence explicitly: probe installs but silently never fires. Replace vague pronoun and jargon with concrete referents. Co-Authored-By: Claude <[email protected]>

RubyVM::InstructionSequence is needed to find the API docs. "Compiles source to bytecode without executing it" explains what they do without assuming MRI internals knowledge. Co-Authored-By: Claude <[email protected]>

p-datadog · 2026-04-22T16:31:22Z

@codex verify that integration tests properly require that iseqs are garbage collected and that this is clearly stated in code comments

chatgpt-codex-connector · 2026-04-22T16:37:07Z

Summary

Clarified the GC dependency in the per-method ISeq integration test comments, explicitly documenting that GC of the require-produced :top ISeq is a hard precondition and that the spec validates this precondition before running assertions.
Clarified the compile_file E2E test comments to state that the scenario intentionally depends on GC, and that :top-ISeq absence plus method-ISeq presence are required preconditions to avoid false positives via whole-file matching.
Committed these changes on the current branch (d526df96) and created a follow-up PR via the make_pr tool as requested.

Testing

✅ bundle exec rake test:di:di_with_ext 2>&1 | tee /tmp/full_rspec.log | grep -E 'Pending:|Failures:|Finished' -A 99 (passed: 751 examples, 0 failures, 19 pending)

View task →

p-datadog · 2026-04-23T15:43:02Z

@codex read the documentation changes in this PR, then improve the documentation:

1- make customer-targeted docs clearly describe any limitations per ruby version

2- shorten added sections to 1/3 of their current size

3- ensure there is an understandable overview of how the system works (for the development guide) and what the customer-visible behavior is (for user guide). delete low level details

chatgpt-codex-connector · 2026-04-23T15:45:17Z

Summary

Reworked the customer guide’s pre-loaded-file section to focus on customer-visible behavior first (what works, what may not fire, and what becomes pending), and made Ruby-version limitations explicit for Ruby 3.1+ vs Ruby 2.6–3.0. docs/DynamicInstrumentation.mdL217-L245
Shortened the newly added documentation substantially by removing low-level internals and replacing them with concise, audience-appropriate guidance in both docs files (including deletion of detailed tables and deep runtime internals from the development guide). docs/DynamicInstrumentation.mdL217-L245 docs/DynamicInstrumentationDevelopment.mdL3-L39
Rewrote the development guide section into a compact, understandable system overview: tracker data flow, GC impact, fallback behavior, Ruby-version known limits, and the GC precondition expected in integration tests. docs/DynamicInstrumentationDevelopment.mdL5-L39
Committed the changes as f90b4fe8 and created a follow-up PR using the make_pr tool.

Testing

✅ git diff -- docs/DynamicInstrumentation.md docs/DynamicInstrumentationDevelopment.md
✅ nl -ba docs/DynamicInstrumentation.md | sed -n '210,290p'; nl -ba docs/DynamicInstrumentationDevelopment.md | sed -n '1,120p'

View task →

p-datadog · 2026-04-23T15:54:18Z

None of the AIs are able to write sensible docs for this PR, I'll tackle that some other time.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b801292f82

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Strech

I would suggest converting doubles in tests into instance_double to avoid over-stabbing

Address review comment: when multiple per-method iseqs contain the same target line (e.g. a method and an inline block sharing a line), picking the first match would silently bind the probe to only one context. Changed .find to .select with a multi-match guard that raises MultiplePathsMatch so the probe is recorded as failed with a clear error message. Also changed resolve_path_suffix to raise MultiplePathsMatch (instead of returning nil) when multiple paths match, consistent with iseqs_for_path_suffix behavior. - Fixed in lib/datadog/di/code_tracker.rb:292 (ambiguous iseq guard) - Fixed in lib/datadog/di/code_tracker.rb:361 (ambiguous path suffix) Co-Authored-By: Claude <[email protected]>

@Strech

Address review comments: - Convert all iseq double() to instance_double(RubyVM::InstructionSequence) to verify stubs against the real interface (per @Strech suggestion) - Add test for ambiguous multi-iseq line match raising MultiplePathsMatch - Add test for ambiguous per-method path suffix raising MultiplePathsMatch Co-Authored-By: Claude <[email protected]>

p-datadog · 2026-04-25T22:23:28Z

Good suggestion — converted all iseq doubles to instance_double(RubyVM::InstructionSequence, ...) across the spec. The stubbed methods (absolute_path, first_lineno, trace_points) are now verified against the real interface.

p-datadog added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Mar 24, 2026

p-datadog changed the base branch from di-iseq-backfill to master March 25, 2026 00:28

github-actions Bot added the profiling Involves Datadog profiling label Mar 25, 2026

p-datadog mentioned this pull request Mar 25, 2026

DI: backfill CodeTracker registry with iseqs for pre-loaded files #5496

Merged

p-datadog force-pushed the di-per-method-iseq branch from 9012b29 to dee647b Compare March 27, 2026 23:45

p-datadog changed the base branch from master to di-iseq-backfill March 27, 2026 23:45

p-datadog changed the base branch from di-iseq-backfill to master March 27, 2026 23:46

p-datadog mentioned this pull request Mar 28, 2026

IGNORE: base for enablement #5524

Closed

p-datadog mentioned this pull request Mar 28, 2026

IGNORE: base for #5525 #5526

Closed

p-datadog force-pushed the di-per-method-iseq branch from dee647b to 4107334 Compare March 28, 2026 05:30

p-datadog changed the base branch from master to di-iseq-backfill March 30, 2026 12:42

p-datadog force-pushed the di-per-method-iseq branch from 873a846 to af1e513 Compare March 30, 2026 12:43

p-datadog changed the base branch from di-iseq-backfill to master March 30, 2026 12:44

p-datadog mentioned this pull request Apr 1, 2026

IGNORE: base for #6035 + #5908 + #6043 #5540

Draft

p-datadog force-pushed the di-per-method-iseq branch 4 times, most recently from c8f8bf9 to 31c6a23 Compare April 8, 2026 18:49

p-datadog force-pushed the di-per-method-iseq branch 4 times, most recently from 22e7d9c to bc3339c Compare April 8, 2026 21:19

p-ddsign and others added 2 commits April 22, 2026 09:49

Fix stale comment: per-method iseq support is implemented

accbd44

The backfill_registry docstring said "per-method iseqs require instrumenter changes and will be supported in a follow-up." This PR is that follow-up — the comment is now stale. Co-Authored-By: Claude <[email protected]>

p-datadog force-pushed the di-per-method-iseq branch from 21af9d4 to dc8db65 Compare April 22, 2026 13:49

p-ddsign and others added 6 commits April 22, 2026 12:05

Remove duplicate di:bootsnap Matrixfile entry

fca5426

The second entry (with jruby) overwrote the first. Keep only the entry without jruby, matching the reviewer's suggestion and the format of adjacent entries. Co-Authored-By: Claude <[email protected]>

Clarify customer impact in Ruby < 3.1 limitation comment

69dd7ec

State the consequence explicitly: probe installs but silently never fires. Replace vague pronoun and jargon with concrete referents. Co-Authored-By: Claude <[email protected]>

Add full class path and description for compile_file/compile

61ad28d

RubyVM::InstructionSequence is needed to find the API docs. "Compiles source to bytecode without executing it" explains what they do without assuming MRI internals knowledge. Co-Authored-By: Claude <[email protected]>

test(di): clarify GC preconditions in iseq integration specs

0cffbb4

p-datadog marked this pull request as ready for review April 23, 2026 15:57

p-datadog requested review from a team as code owners April 23, 2026 15:57

p-datadog force-pushed the di-per-method-iseq branch from b801292 to 0cffbb4 Compare April 23, 2026 16:00

chatgpt-codex-connector Bot reviewed Apr 23, 2026

View reviewed changes

Comment thread lib/datadog/di/code_tracker.rb Outdated

Comment thread lib/datadog/di/code_tracker.rb Outdated

p-datadog mentioned this pull request Apr 23, 2026

DI: document pre-loaded file line probe behavior #5637

Closed

Strech approved these changes Apr 24, 2026

View reviewed changes

p-ddsign and others added 2 commits April 25, 2026 18:19

p-datadog merged commit faaec08 into master Apr 27, 2026
865 of 867 checks passed

p-datadog deleted the di-per-method-iseq branch April 27, 2026 11:16

dd-octo-sts Bot added this to the 2.32.0 milestone Apr 27, 2026

p-datadog removed the profiling Involves Datadog profiling label May 7, 2026

marcotc mentioned this pull request May 7, 2026

Bump to version 2.32.0 #5710

Merged

Uh oh!

Conversation

p-datadog commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Uh oh!

github-actions Bot commented Mar 27, 2026 • edited by dd-octo-sts Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Typing analysis

steep:ignore comments

Untyped methods

Uh oh!

datadog-prod-us1-6 Bot commented Mar 28, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

p-datadog commented Apr 8, 2026

Uh oh!

p-datadog commented Apr 22, 2026

Uh oh!

chatgpt-codex-connector Bot commented Apr 22, 2026

Summary

Uh oh!

p-datadog commented Apr 23, 2026

Uh oh!

chatgpt-codex-connector Bot commented Apr 23, 2026

Summary

Uh oh!

p-datadog commented Apr 23, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Strech left a comment

Choose a reason for hiding this comment

Uh oh!

p-datadog commented Apr 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

p-datadog commented Mar 24, 2026 •

edited

Loading

pr-commenter Bot commented Mar 25, 2026 •

edited

Loading

github-actions Bot commented Mar 27, 2026 •

edited by dd-octo-sts Bot

Loading

`steep:ignore` comments

datadog-prod-us1-6 Bot commented Mar 28, 2026 •

edited by datadog-official Bot

Loading