Skip to content

Symbol database: stop invoking application-overridable reflection during extraction#5946

Merged
p-datadog merged 3 commits into
masterfrom
symdb-extraction-avoid-customer-code
Jun 25, 2026
Merged

Symbol database: stop invoking application-overridable reflection during extraction#5946
p-datadog merged 3 commits into
masterfrom
symdb-extraction-avoid-customer-code

Conversation

@p-datadog

Copy link
Copy Markdown
Member

What does this PR do?

Stops symbol database extraction from calling application-overridable reflection methods directly. Extraction now dispatches name, superclass, included_modules, ancestors, class, is_a?, const_get, constants, class_variables, and autoload? through cached unbound Module/Class/Kernel methods (the precedent already set by MODULE_NAME / safe_mod_name), routes every name lookup through safe_mod_name, and replaces obj.is_a?(Klass) checks with Klass === obj. resolve_scope_type now walks the fully-qualified name segment by segment with autoload guards instead of Object.const_get, so it classifies CLASS vs MODULE without triggering autoloads.

Motivation:

Extraction introspects arbitrary application objects. Calling these methods directly executes application code during a passive introspection pass: an overridden method can return wrong data, run customer code as a side effect, or raise, and const_get/Object.const_get can autoload (loading customer code) while merely classifying a constant. safe_mod_name already guarded the primary module-name lookup; this extends the same protection to the remaining sites.

Change log entry

None.

Additional Notes:

  • Independent of the catch-all-rescue hardening in DEBUG-5089 Rescue all exceptions in DI and SymDB #5945; both branch from master.
  • Behaviour note: a constant whose value is a BasicObject is now classified (type: "BasicObject") instead of being dropped, because the real class is read via an unbound Kernel#class.

How to test the change?

  • bundle exec rake standard typecheck is clean.
  • New specs under spec/datadog/symbol_database/extractor_spec.rb ("immunity to customer-overridden reflection") assert each site reads real data and avoids side effects when an application overrides reflection, including the autoload-safe classification path.
  • bundle exec rspec spec/datadog/symbol_database — 372 examples, 0 failures.

@p-datadog p-datadog added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Jun 23, 2026
@dd-octo-sts dd-octo-sts Bot added the debugger Live Debugger (+Dynamic Instrumentation, +Symbol Database) label Jun 23, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Typing analysis

Note: Ignored files are excluded from the next sections.

steep:ignore comments

This PR introduces 12 steep:ignore comments, and clears 13 steep:ignore comments.

steep:ignore comments (+12-13)Introduced:
lib/datadog/symbol_database/extractor.rb:355
lib/datadog/symbol_database/extractor.rb:367
lib/datadog/symbol_database/extractor.rb:382
lib/datadog/symbol_database/extractor.rb:639
lib/datadog/symbol_database/extractor.rb:669
lib/datadog/symbol_database/extractor.rb:790
lib/datadog/symbol_database/extractor.rb:867
lib/datadog/symbol_database/extractor.rb:885
lib/datadog/symbol_database/extractor.rb:886
lib/datadog/symbol_database/extractor.rb:976
lib/datadog/symbol_database/extractor.rb:977
lib/datadog/symbol_database/extractor.rb:1025
Cleared:
lib/datadog/symbol_database/extractor.rb:335
lib/datadog/symbol_database/extractor.rb:347
lib/datadog/symbol_database/extractor.rb:361
lib/datadog/symbol_database/extractor.rb:506
lib/datadog/symbol_database/extractor.rb:617
lib/datadog/symbol_database/extractor.rb:647
lib/datadog/symbol_database/extractor.rb:768
lib/datadog/symbol_database/extractor.rb:845
lib/datadog/symbol_database/extractor.rb:863
lib/datadog/symbol_database/extractor.rb:864
lib/datadog/symbol_database/extractor.rb:943
lib/datadog/symbol_database/extractor.rb:944
lib/datadog/symbol_database/extractor.rb:992

Untyped methods

This PR introduces 1 untyped method and 4 partially typed methods, and clears 1 untyped method and 4 partially typed methods.

Untyped methods (+1-1)Introduced:
sig/datadog/symbol_database/extractor.rbs:24
└── def initialize: (logger: untyped, settings: untyped) -> void
Cleared:
sig/datadog/symbol_database/extractor.rbs:15
└── def initialize: (logger: untyped, settings: untyped) -> void
Partially typed methods (+4-4)Introduced:
sig/datadog/symbol_database/extractor.rbs:39
└── def convert_tree_to_scope: (::String file_path, untyped root) -> Scope
sig/datadog/symbol_database/extractor.rbs:60
└── def build_class_language_specifics: (Class klass) -> Hash[::Symbol, untyped]
sig/datadog/symbol_database/extractor.rbs:74
└── def place_in_tree: (untyped root, Array[String] name_parts, Module mod, String mod_name, Array[untyped] methods, String file_path) -> void
sig/datadog/symbol_database/extractor.rbs:78
└── def convert_node_to_scope: (untyped node) -> Scope
Cleared:
sig/datadog/symbol_database/extractor.rbs:30
└── def convert_tree_to_scope: (::String file_path, untyped root) -> Scope
sig/datadog/symbol_database/extractor.rbs:51
└── def build_class_language_specifics: (Class klass) -> Hash[::Symbol, untyped]
sig/datadog/symbol_database/extractor.rbs:65
└── def place_in_tree: (untyped root, Array[String] name_parts, Module mod, String mod_name, Array[untyped] methods, String file_path) -> void
sig/datadog/symbol_database/extractor.rbs:69
└── def convert_node_to_scope: (untyped node) -> Scope

Untyped other declarations

This PR introduces 2 untyped other declarations, and clears 2 untyped other declarations. It increases the percentage of typed other declarations from 82.63% to 82.71% (+0.08%).

Untyped other declarations (+2-2)Introduced:
sig/datadog/symbol_database/extractor.rbs:22
└── @logger: untyped
sig/datadog/symbol_database/extractor.rbs:23
└── @settings: untyped
Cleared:
sig/datadog/symbol_database/extractor.rbs:13
└── @logger: untyped
sig/datadog/symbol_database/extractor.rbs:14
└── @settings: untyped

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jun 23, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 20.41%
Overall Coverage: 90.01% (-0.04%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0210296 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jun 23, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-25 19:51:16

Comparing candidate commit 0210296 in PR branch symdb-extraction-avoid-customer-code with baseline commit 05d39cb in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

@p-datadog
p-datadog marked this pull request as ready for review June 24, 2026 21:45
@p-datadog
p-datadog requested a review from Copilot June 24, 2026 21:45
@p-datadog
p-datadog requested a review from a team as a code owner June 24, 2026 21:45
@p-datadog

Copy link
Copy Markdown
Member Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Datadog::SymbolDatabase::Extractor so symbol database extraction avoids calling application-overridable reflection methods directly, reducing the risk of side effects, incorrect metadata, or autoload-triggered code execution during passive introspection.

Changes:

  • Dispatches core reflection APIs (name, superclass, included_modules, ancestors, constants, const_get, const_defined?, autoload?, etc.) through cached UnboundMethods and routes name lookups through safe_mod_name.
  • Updates scope classification (resolve_scope_type) to walk namespaces segment-by-segment with autoload guards instead of Object.const_get.
  • Adds specs covering “hostile” overrides of reflection methods and updates the RBS signature to include the new cached UnboundMethod constants.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
lib/datadog/symbol_database/extractor.rb Uses cached unbound reflection methods + safer scope-type resolution to avoid application overrides/autoload side effects during extraction.
sig/datadog/symbol_database/extractor.rbs Adds RBS declarations for the newly introduced cached UnboundMethod constants.
spec/datadog/symbol_database/extractor_spec.rb Adds regression coverage ensuring extraction is immune to customer-overridden reflection and avoids autoload triggering.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/datadog/symbol_database/extractor.rb
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: f32ae41f63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

p-ddsign added 3 commits June 25, 2026 15:25
Extraction walks arbitrary application objects via reflection. Calling
name/superclass/included_modules/ancestors/class/is_a?/const_get/constants/
autoload? directly executes application code (and, for const_get, can autoload
as a side effect): an overridden method can return wrong data, run customer
code during a passive introspection pass, or raise.

Cache UnboundMethods for these and dispatch them explicitly (the precedent set
by MODULE_NAME / safe_mod_name), route every name lookup through safe_mod_name,
and replace is_a? checks with Klass === obj. resolve_scope_type now walks the
FQN segment by segment with autoload guards instead of Object.const_get, so it
classifies CLASS vs MODULE without triggering autoloads.
Cover that build_class_language_specifics, extract_scope_symbols,
resolve_scope_type, and extract read real data and avoid side effects when an
application overrides name/superclass/included_modules/ancestors/class/is_a?/
const_get/constants, including the autoload-safe classification path.
Wrap the `Class === mod` / `Class === current` ternary conditions in
parentheses. Replacing `mod.is_a?(Class) ? ...` with `Class === mod ? ...`
introduced complex (binary-operator) ternary conditions, which StandardRB's
Style/TernaryParentheses cop requires to be parenthesized. The is_a? form was
a method call and did not trigger the cop.

This was failing the standard/lint CI check at extractor.rb:920, 925, 954.

Verified: `bundle exec standardrb` exits 0 (clean).
@p-datadog
p-datadog force-pushed the symdb-extraction-avoid-customer-code branch from f32ae41 to 0210296 Compare June 25, 2026 19:25
@p-datadog
p-datadog merged commit 7a33a61 into master Jun 25, 2026
587 checks passed
@p-datadog
p-datadog deleted the symdb-extraction-avoid-customer-code branch June 25, 2026 20:09
@dd-octo-sts dd-octo-sts Bot added this to the 2.37.0 milestone Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos debugger Live Debugger (+Dynamic Instrumentation, +Symbol Database)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants