Skip to content

[DI] Implement PII redaction#5053

Merged
watson merged 4 commits into
masterfrom
watson/DEBUG-2630/add-redaction
Jan 8, 2025
Merged

[DI] Implement PII redaction#5053
watson merged 4 commits into
masterfrom
watson/DEBUG-2630/add-redaction

Conversation

@watson

@watson watson commented Dec 20, 2024

Copy link
Copy Markdown
Collaborator

The algorithm will look for:

  • names of variables
  • names of object properties
  • names of keys in maps

The names will be matched against a disallow-list and if a match is found, its value will be redacted.

The list is hardcoded and can be found here:

packages/dd-trace/src/debugger/devtools_client/snapshot/redaction.js

It's possible to add names to the list using the following environment variable:

DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS

Or it's possble to remove names from the list using the following environment variable:

DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS

Each environment variable takes a list of names separated by commas.

Support for redacting instances of specific classes is not included in this PR.

Blocked by

@watson
watson requested review from a team as code owners December 20, 2024 18:46
@watson watson self-assigned this Dec 20, 2024
@watson watson added semver-minor debugger Dynamic Instrumentation & Live Debugger labels Dec 20, 2024

watson commented Dec 20, 2024

Copy link
Copy Markdown
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions

github-actions Bot commented Dec 20, 2024

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 8.44 MB
Deduped: 94.78 MB
No deduping: 95.3 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.3.0 | 29.43 MB | 29.43 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@pr-commenter

pr-commenter Bot commented Dec 20, 2024

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-01-08 07:32:19

Comparing candidate commit 3936cee in PR branch watson/DEBUG-2630/add-redaction with baseline commit e2bee27 in branch master.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 774 metrics, 22 unstable metrics.

scenario:log-without-log-20

  • 🟥 cpu_user_time [+19.928ms; +24.910ms] or [+5.804%; +7.255%]
  • 🟥 execution_time [+20.471ms; +22.438ms] or [+5.247%; +5.751%]

juan-fernandez
juan-fernandez previously approved these changes Jan 2, 2025
@watson
watson force-pushed the watson/DEBUG-2630/add-redaction branch from ac6401b to 883b932 Compare January 6, 2025 11:34
@watson
watson requested a review from a team as a code owner January 6, 2025 11:34
@watson
watson requested a review from vitor-de-araujo January 6, 2025 11:34
juan-fernandez
juan-fernandez previously approved these changes Jan 7, 2025
@watson
watson enabled auto-merge (squash) January 8, 2025 05:07
watson added 3 commits January 8, 2025 06:21
The algorithm will look for:
- names of variables
- names of object properties
- names of keys in maps

The names will be matched against a disallow-list and if a match is
found, its value will be redacted.

The list is hardcoded and can be found here:

    packages/dd-trace/src/debugger/devtools_client/snapshot/redaction.js

It's possible to add names to the list using the following environment
variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS

Or it's possble to remove names from the list using the following
environment variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS

Each environment variable takes a list of names separated by commas.

Support for redacting instances of specific classes is not included in
this commit.
@watson
watson merged commit b36ce05 into master Jan 8, 2025
@watson
watson deleted the watson/DEBUG-2630/add-redaction branch January 8, 2025 09:32
watson added a commit that referenced this pull request Jan 22, 2025
The algorithm will look for:
- names of variables
- names of object properties
- names of keys in maps

The names will be matched against a disallow-list and if a match is
found, its value will be redacted.

The list is hardcoded and can be found here:

    packages/dd-trace/src/debugger/devtools_client/snapshot/redaction.js

It's possible to add names to the list using the following environment
variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS

Or it's possible to remove names from the list using the following
environment variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS

Each environment variable takes a list of names separated by commas.

Support for redacting instances of specific classes is not included in
this commit.
@watson watson mentioned this pull request Jan 22, 2025
watson added a commit that referenced this pull request Jan 22, 2025
The algorithm will look for:
- names of variables
- names of object properties
- names of keys in maps

The names will be matched against a disallow-list and if a match is
found, its value will be redacted.

The list is hardcoded and can be found here:

    packages/dd-trace/src/debugger/devtools_client/snapshot/redaction.js

It's possible to add names to the list using the following environment
variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS

Or it's possible to remove names from the list using the following
environment variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS

Each environment variable takes a list of names separated by commas.

Support for redacting instances of specific classes is not included in
this commit.
@watson watson mentioned this pull request Jan 22, 2025
watson added a commit that referenced this pull request Jan 23, 2025
The algorithm will look for:
- names of variables
- names of object properties
- names of keys in maps

The names will be matched against a disallow-list and if a match is
found, its value will be redacted.

The list is hardcoded and can be found here:

    packages/dd-trace/src/debugger/devtools_client/snapshot/redaction.js

It's possible to add names to the list using the following environment
variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS

Or it's possible to remove names from the list using the following
environment variable:

    DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS

Each environment variable takes a list of names separated by commas.

Support for redacting instances of specific classes is not included in
this commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debugger Dynamic Instrumentation & Live Debugger semver-minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants