APMSP-3553 DI: redactor normalize Rack CGI header prefix#5911
Conversation
`Redactor#normalize` previously stripped only `[-_$@]` before matching against `DEFAULT_REDACTED_IDENTIFIERS`. Hash keys originating from a Rack environment use the CGI form `HTTP_<HEADER>` (Rack rewrites incoming HTTP headers, converting dashes to underscores and prepending `HTTP_`). Those keys collapsed to `httpauthorization`, `httpcookie`, `httpxapikey`, etc. — none of which appear in the default list — so the corresponding values were not matched against entries like `authorization`, `cookie`, or `xapikey`. Any snapshot or line probe placed where a Rack `env` hash is in lexical scope (every Rack-based middleware, Rails controller, or Sinatra handler) emitted those values unredacted. Strip a leading `http_` in `normalize` before the punctuation gsub. The sub runs after `downcase`, so it only matches the literal Rack CGI form — identifiers that happen to start with the letters `http` without a following underscore (e.g. `httpinfo`) are unaffected. This generalizes over every entry in the default list, so future additions do not need a parallel `HTTP_`-prefixed twin maintained in lockstep. Specs added to `spec/datadog/di/redactor_spec.rb` cover the canonical Rack header forms (`HTTP_AUTHORIZATION`, `HTTP_COOKIE`, `HTTP_X_API_KEY`, `HTTP_X_AUTH_TOKEN`, `HTTP_SET_COOKIE`), a non-redacted control (`HTTP_REFERER`, `HTTP_HOST`), the `http`-without- underscore boundary case, and the user-defined-list interaction (a user-supplied identifier prefixed with `http_` matches both the CGI form and the bare form). Verified: - `bundle exec rspec spec/datadog/di/redactor_spec.rb` 49 examples, 0 failures - `bundle exec rake test:di:di_with_ext` 808 examples, 0 failures, 8 pending (pre-existing) - `bundle exec rake typecheck` No type error detected - `bundle exec rake standard` No offenses
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 5511be4 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-06-26 19:37:52 Comparing candidate commit 5511be4 in PR branch Found 1 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 1 unstable metrics.
|
The previous commit makes Redactor#normalize strip a leading 'http_' so Rack CGI-form keys (HTTP_AUTHORIZATION, ...) match the default redaction list under their bare names. Because the same normalize runs over redaction_excluded_identifiers, excluding a bare name symmetrically suppresses redaction of its HTTP_-prefixed Rack form — but the prior spec only covered the bare-form exclusion. Add a spec case in the 'when excluded identifiers are specified' context that excludes 'password' and asserts HTTP_PASSWORD is also not redacted, locking down the symmetry the PR description claims. bundle exec rspec spec/datadog/di/redactor_spec.rb 50 examples, 0 failures
The comment attributed the httpinfo exclusion to running the strip before the punctuation gsub. That protection actually comes from the \Ahttp_ regex requiring an underscore after http. The real reason for the ordering is that the gsub removes the underscore, so a later \Ahttp_ match would never fire. Reword the comment to state both facts accurately.
|
@codex review |
There was a problem hiding this comment.
Pull request overview
This PR improves Dynamic Instrumentation (DI) snapshot redaction by updating Redactor#normalize to treat Rack CGI-style header keys (HTTP_<HEADER>) as equivalent to their bare header names, ensuring default sensitive identifiers (e.g., authorization, cookie, xapikey) correctly redact values captured from a Rack env.
Changes:
- Update
Redactor#normalizeto strip a leadinghttp_(after downcasing) before applying the existing punctuation removal. - Add RSpec coverage for Rack CGI header keys redacting the same as their bare counterparts, including boundary/control cases and user-defined identifier behavior.
- Add coverage that the same normalization applies to the exclusion list (e.g., excluding
passwordalso excludesHTTP_PASSWORD).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
lib/datadog/di/redactor.rb |
Adjusts identifier normalization to collapse Rack CGI HTTP_ header keys to the same canonical form used by default/user redaction lists. |
spec/datadog/di/redactor_spec.rb |
Adds targeted tests validating redaction/exclusion behavior for HTTP_* keys and boundary cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Co-authored-by: Sergey Fedorov <[email protected]>
What does this PR do?
Makes
Redactor#normalizestrip a leadinghttp_so that Rack CGIheader keys (
HTTP_AUTHORIZATION,HTTP_COOKIE,HTTP_X_API_KEY,…) collapse to the same identifier as the bare header name and are
matched against
DEFAULT_REDACTED_IDENTIFIERS(authorization,cookie,xapikey, …).Motivation
Rack rewrites incoming HTTP header names to the CGI form
HTTP_<HEADER>(dashes to underscores,
HTTP_prefix). The previousnormalizeimplementation stripped only
[-_$@], so Rack-shaped keys collapsed tohttpauthorization,httpcookie,httpxapikey, etc., and no entry inthe default identifier list matched them. Any hash capture that
included a Rack
env(every Rack-based middleware, Rails controller,or Sinatra handler keeps it in lexical scope) ended up emitting those
values verbatim into the snapshot, while the same headers under their
bare names (
Authorization,Cookie,X-API-Key) were redacted.Snapshots should not distinguish between
env["HTTP_AUTHORIZATION"]and a local
authorization— both name the same thing.The change is in
normalizerather thanDEFAULT_REDACTED_IDENTIFIERSbecause the list is shared across tracers and the gap is structural:
every entry in the list has an
HTTP_-prefixed twin under Rack, andmaintaining the twin in lockstep is more error-prone than collapsing
the prefix at normalization time. The strip runs after
downcaseandis anchored to
\Ahttp_, so identifiers that simply start with theletters
httpwithout a following underscore (e.g.httpinfo) areunaffected. User-supplied identifiers go through the same path, so a
user-defined
http_customentry now also matches the barecustomform — the same collapsing already applies to the punctuation gsub
(
pass_wordmatchespassword), and this extends the principle tothe Rack prefix.
Change log entry
Yes. Dynamic Instrumentation: redactor normalizes the Rack CGI header
prefix (
HTTP_<HEADER>), so snapshot captures of a Rack environmentredact sensitive header values under the same defaults as their bare
header names.
Implementation
Redactor#normalizebecomes:The
subruns once at normalization time; there is no per-lookupcost beyond a single anchored regex check on already-downcased input.
Both the default list and the user-defined / excluded lists are
normalized with the same function (see the
redacted_identifiersprivate method), so the prefix strip applies symmetrically — a name
excluded via
redaction_excluded_identifierscontinues to suppressredaction regardless of whether the captured key is the bare form or
the
HTTP_-prefixed form.How to test the change?
49 examples, 0 failures.
808 examples, 0 failures, 8 pending (same 8 Ruby-version-gated pending
tests as on master).
New spec cases live under
describe '#redact_identifier?'and cover:HTTP_AUTHORIZATION,HTTP_COOKIE,HTTP_X_API_KEY,HTTP_X_AUTH_TOKEN,HTTP_SET_COOKIE— all match their barecounterparts in
DEFAULT_REDACTED_IDENTIFIERSHTTP_REFERER,HTTP_HOST— Rack-shaped but not in the list;pass through unredacted as controls
httpinfo— boundary case proving the prefix strip is anchored tothe literal CGI form (leading
http_), not arbitrary identifiersthat begin with the letters
httpHTTP_CUSTOMandcustomagainst a user-definedhttp_customentry — confirms the user-defined list normalizes through the same
function