Skip to content

fix(aws-sdk): record kinesis DSM checkpoint only when context ships#9183

Merged
tlhunter merged 8 commits into
masterfrom
BridgeAR/2026-07-02-kinesis-dsm-checkpoint-size-gate
Jul 14, 2026
Merged

fix(aws-sdk): record kinesis DSM checkpoint only when context ships#9183
tlhunter merged 8 commits into
masterfrom
BridgeAR/2026-07-02-kinesis-dsm-checkpoint-size-gate

Conversation

@BridgeAR

@BridgeAR BridgeAR commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

When a Kinesis record exceeds the 1 MiB cap, injectToMessage bailed after
setDSMCheckpoint had already run: it recorded a produce checkpoint, advanced the
pathway context, and tagged pathway.hash on the span, yet params.Data was never
written so nothing shipped on the wire. The producer left a dangling pathway edge no
consumer completes and a span tagged with a hash for a message it never sent.

The fix establishes the invariant a DSM checkpoint is recorded iff the context
actually lands on the wire
: gate on the trace-context payload first, and only call
setDSMCheckpoint + DsmPathwayCodec.encode + write params.Data for a record that
clears the gate.

Why

Sizing the DSM-inclusive payload needs the encoded pathway, which needs the checkpoint —
the side effect we defer. The record is gated on the trace-context-inclusive payload and
the bounded dd-pathway-ctx-base64 bytes are added after the gate. They cannot
realistically tip a record that already cleared 1 MiB with room for them, so the common
case measures the on-wire size correctly and the checkpoint stays honest.

The non-DSM path keeps its single JSON.stringify; the second serialize only fires on the
DSM path once a record has passed the gate.

@dd-octo-sts

dd-octo-sts Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.71 MB
Deduped: 7.37 MB
No deduping: 7.37 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.1 | 122.62 kB | 437.94 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@pr-commenter

pr-commenter Bot commented Jul 2, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-10 20:25:35

Comparing candidate commit 9d7589d in PR branch BridgeAR/2026-07-02-kinesis-dsm-checkpoint-size-gate with baseline commit f542182 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2320 metrics, 38 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-191.117ms; +125.387ms] or [-7.270%; +4.770%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-233.114ms; +236.029ms] or [-9.165%; +9.280%]

scenario:appsec-appsec-enabled-with-attacks-24

  • unstable execution_time [-160.971ms; +148.880ms] or [-5.230%; +4.837%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-185.488ms; +140.112ms] or [-6.455%; +4.876%]

scenario:appsec-control-20

  • unstable execution_time [-130.745ms; +125.717ms] or [-7.980%; +7.673%]

scenario:appsec-control-24

  • unstable execution_time [-119.534ms; +96.633ms] or [-9.731%; +7.867%]

scenario:appsec-control-26

  • unstable execution_time [-129.542ms; +123.537ms] or [-10.542%; +10.054%]

scenario:appsec-iast-no-vulnerability-control-20

  • unstable execution_time [-22.433ms; +11.276ms] or [-8.705%; +4.376%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-20

  • unstable execution_time [-13250.571µs; +13329.412µs] or [-5.086%; +5.116%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-20

  • unstable execution_time [-31.665ms; +35.344ms] or [-5.669%; +6.327%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-20

  • unstable execution_time [-28.343ms; +33.760ms] or [-5.103%; +6.078%]

scenario:debugger-line-probe-with-snapshot-default-24

  • unstable cpu_user_time [-1779.947ms; +2678.153ms] or [-21.536%; +32.404%]
  • unstable execution_time [-1772.896ms; +2666.922ms] or [-19.723%; +29.669%]
  • unstable instructions [-15.4G instructions; +22.8G instructions] or [-22.743%; +33.841%]
  • unstable max_rss_usage [-7.077MB; +11.174MB] or [-4.515%; +7.129%]
  • unstable throughput [-750.829op/s; +514.471op/s] or [-20.617%; +14.127%]

scenario:debugger-line-probe-with-snapshot-default-26

  • unstable cpu_user_time [-2.852s; +0.380s] or [-28.494%; +3.801%]
  • unstable execution_time [-2.912s; +0.412s] or [-27.139%; +3.842%]
  • unstable instructions [-25.0G instructions; +3.3G instructions] or [-29.880%; +3.960%]
  • unstable throughput [-92.945op/s; +625.122op/s] or [-2.980%; +20.044%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-1812.319ms; +3727.372ms] or [-20.297%; +41.745%]
  • unstable execution_time [-1888.976ms; +3843.098ms] or [-19.586%; +39.848%]
  • unstable instructions [-15.1G instructions; +31.5G instructions] or [-20.590%; +43.023%]
  • unstable max_rss_usage [-7.081MB; +14.650MB] or [-4.461%; +9.229%]
  • unstable throughput [-1051.877op/s; +534.936op/s] or [-30.478%; +15.500%]

scenario:debugger-line-probe-without-snapshot-26

  • unstable cpu_user_time [-2.822s; +0.378s] or [-28.347%; +3.801%]
  • unstable execution_time [-2.835s; +0.365s] or [-26.490%; +3.408%]
  • unstable instructions [-24.9G instructions; +3.4G instructions] or [-29.803%; +4.071%]
  • unstable throughput [-80.557op/s; +621.828op/s] or [-2.583%; +19.940%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-483.990ms; +239.125ms] or [-10.007%; +4.944%]
  • unstable execution_time [-484.696ms; +241.195ms] or [-9.869%; +4.911%]
  • unstable throughput [-80614.844op/s; +169238.313op/s] or [-4.713%; +9.894%]

scenario:fs-subscribed-24

  • unstable execution_time [-29.154ms; +35.771ms] or [-7.175%; +8.804%]
  • unstable throughput [-127365.561op/s; +102442.359op/s] or [-5.739%; +4.616%]

scenario:plugin-claude-agent-sdk-delayed-noisy-hook-indexed-24

  • unstable execution_time [-13.891ms; +25.329ms] or [-4.406%; +8.034%]

scenario:plugin-graphql-long-with-depth-on-max-20

  • unstable cpu_user_time [-722.863ms; +716.339ms] or [-5.616%; +5.565%]
  • unstable execution_time [-727.694ms; +718.578ms] or [-5.531%; +5.462%]
  • unstable throughput [-3.245op/s; +3.344op/s] or [-5.306%; +5.469%]

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Jul 2, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 96.54% (+0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9d7589d | Docs | Datadog PR Page | Give us feedback!

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.58%. Comparing base (f542182) to head (9d7589d).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #9183    +/-   ##
========================================
  Coverage   96.57%   96.58%            
========================================
  Files         918      918            
  Lines      121564   121643    +79     
  Branches    20380    21041   +661     
========================================
+ Hits       117399   117484    +85     
+ Misses       4165     4159     -6     
Flag Coverage Δ
aiguard 53.56% <43.47%> (-0.11%) ⬇️
aiguard-integration 57.04% <72.47%> (+0.03%) ⬆️
apm-bucket-0 53.79% <43.47%> (-0.04%) ⬇️
apm-bucket-1 58.12% <69.56%> (-0.01%) ⬇️
apm-bucket-2 56.11% <43.47%> (-0.05%) ⬇️
apm-capabilities-tracing 60.35% <75.39%> (+0.03%) ⬆️
apm-integrations-aerospike 52.94% <43.47%> (-0.04%) ⬇️
apm-integrations-confluentinc-kafka-javascript 57.11% <73.04%> (+0.01%) ⬆️
apm-integrations-couchbase 53.27% <43.47%> (-0.04%) ⬇️
apm-integrations-http 57.99% <69.56%> (-0.01%) ⬇️
apm-integrations-kafkajs 57.76% <73.04%> (-0.02%) ⬇️
apm-integrations-next 54.24% <43.47%> (-0.04%) ⬇️
apm-integrations-prisma 54.17% <43.47%> (-0.04%) ⬇️
apm-integrations-tedious 53.05% <43.47%> (-0.04%) ⬇️
appsec 68.97% <69.56%> (-0.09%) ⬇️
appsec-express_fastify_graphql 65.61% <69.56%> (-0.04%) ⬇️
appsec-integration 45.60% <44.03%> (+0.01%) ⬆️
appsec-kafka_ldapjs_lodash 59.12% <69.56%> (-0.01%) ⬇️
appsec-mongodb-core_mongoose_mysql 62.26% <69.56%> (-0.02%) ⬇️
appsec-next 52.85% <43.47%> (-0.04%) ⬇️
appsec-node-serialize_passport_postgres 61.90% <69.56%> (-0.02%) ⬇️
appsec-sourcing_stripe_template 60.24% <69.56%> (-0.03%) ⬇️
debugger 65.70% <44.03%> (-0.03%) ⬇️
instrumentations-bucket-0 48.77% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-1 54.95% <43.47%> (-0.05%) ⬇️
instrumentations-bucket-10 56.61% <43.47%> (-0.05%) ⬇️
instrumentations-bucket-11 48.76% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-12 48.86% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-13 48.70% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-2 49.75% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-3 53.83% <43.47%> (-0.04%) ⬇️
instrumentations-bucket-4 49.18% <43.47%> (-0.03%) ⬇️
instrumentations-bucket-5 53.06% <69.56%> (+0.01%) ⬆️
instrumentations-bucket-6 55.65% <69.56%> (+<0.01%) ⬆️
instrumentations-bucket-7 53.60% <43.47%> (-0.04%) ⬇️
instrumentations-bucket-8 54.63% <43.47%> (-0.04%) ⬇️
instrumentations-bucket-9 56.09% <43.47%> (-0.04%) ⬇️
instrumentations-instrumentation-couchbase 48.20% <44.03%> (-0.02%) ⬇️
instrumentations-integration-esbuild 34.01% <44.03%> (+0.05%) ⬆️
llmobs-ai_anthropic_bedrock 57.77% <57.06%> (+<0.01%) ⬆️
llmobs-bucket-1 57.02% <57.06%> (+<0.01%) ⬆️
llmobs-openai 57.60% <69.56%> (+<0.01%) ⬆️
llmobs-sdk 59.97% <70.43%> (-0.02%) ⬇️
llmobs-vertex-ai 54.21% <43.47%> (-0.04%) ⬇️
master-coverage 96.58% <100.00%> (?)
openfeature 54.61% <44.03%> (+<0.01%) ⬆️
openfeature-unit 49.85% <44.03%> (-0.03%) ⬇️
platform-core_esbuild_instrumentations-misc 38.48% <44.03%> (+0.01%) ⬆️
platform-integration 62.27% <77.06%> (+0.03%) ⬆️
platform-shimmer_unit-guardrails_webpack 37.32% <44.03%> (+0.01%) ⬆️
plugins-bucket-0 53.24% <57.06%> (+0.02%) ⬆️
plugins-bucket-1 55.13% <71.55%> (+0.05%) ⬆️
plugins-bucket-11 55.60% <43.47%> (-0.04%) ⬇️
plugins-bucket-18 57.12% <73.04%> (+<0.01%) ⬆️
plugins-bucket-19 55.39% <49.56%> (-0.01%) ⬇️
plugins-bucket-20 57.43% <43.47%> (-0.05%) ⬇️
plugins-bucket-4 53.85% <43.47%> (-0.04%) ⬇️
plugins-bullmq_cassandra_cookie 57.44% <73.04%> (+<0.01%) ⬆️
plugins-cookie-parser_crypto_dd-trace-api 52.33% <45.21%> (-0.04%) ⬇️
plugins-fetch_fs_generic-pool 54.59% <69.56%> (+0.08%) ⬆️
plugins-google-cloud-pubsub_grpc_handlebars 59.91% <73.04%> (+<0.01%) ⬆️
plugins-hapi_hono_ioredis 55.86% <43.47%> (-0.05%) ⬇️
plugins-jest_knex_langgraph 51.68% <43.47%> (-0.04%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 53.41% <43.47%> (-0.07%) ⬇️
plugins-lodash_mariadb_memcached 54.21% <43.47%> (-0.04%) ⬇️
plugins-moleculer_mongodb_mongodb-core 57.33% <69.56%> (+<0.01%) ⬆️
plugins-mongoose_multer_mysql 54.71% <43.47%> (-0.04%) ⬇️
plugins-mysql2_nats_node-serialize 56.25% <69.56%> (+<0.01%) ⬆️
plugins-opensearch_passport-http_pino 55.11% <74.78%> (+0.01%) ⬆️
plugins-postgres_process_pug 54.29% <43.47%> (-0.04%) ⬇️
plugins-redis_router_sequelize 57.07% <69.56%> (-0.01%) ⬇️
plugins-test-and-upstream-rhea_undici_url 57.08% <73.04%> (+<0.01%) ⬆️
plugins-valkey_vm_winston 53.86% <49.56%> (-0.03%) ⬇️
plugins-ws 54.85% <70.43%> (+0.01%) ⬆️
profiling 58.19% <44.03%> (-0.05%) ⬇️
serverless-aws-sdk-aws-sdk 50.73% <57.06%> (+0.02%) ⬆️
serverless-aws-sdk-bedrockruntime 50.70% <41.36%> (-0.02%) ⬇️
serverless-aws-sdk-client 52.04% <57.83%> (+0.02%) ⬆️
serverless-aws-sdk-dynamodb 51.77% <41.36%> (-0.02%) ⬇️
serverless-aws-sdk-eventbridge 46.29% <61.46%> (+0.01%) ⬆️
serverless-aws-sdk-kinesis 54.78% <72.25%> (+0.03%) ⬆️
serverless-aws-sdk-lambda 52.85% <57.06%> (+0.02%) ⬆️
serverless-aws-sdk-s3 51.55% <41.36%> (-0.02%) ⬇️
serverless-aws-sdk-serverless-peer-service 54.81% <80.62%> (+0.04%) ⬆️
serverless-aws-sdk-sns 55.54% <67.01%> (+0.01%) ⬆️
serverless-aws-sdk-sqs 55.96% <62.82%> (+0.01%) ⬆️
serverless-aws-sdk-stepfunctions 51.32% <57.06%> (+0.02%) ⬆️
serverless-aws-sdk-util 48.47% <44.03%> (-0.06%) ⬇️
serverless-bucket-0 55.27% <71.55%> (+0.05%) ⬆️
serverless-bucket-1 56.04% <69.56%> (+<0.01%) ⬆️
test-optimization-cucumber 72.92% <71.55%> (-0.04%) ⬇️
test-optimization-cypress 66.30% <44.03%> (-0.01%) ⬇️
test-optimization-jest 73.88% <71.55%> (-0.06%) ⬇️
test-optimization-mocha 74.29% <71.55%> (+0.04%) ⬆️
test-optimization-playwright-playwright-atr 61.55% <44.03%> (-0.03%) ⬇️
test-optimization-playwright-playwright-efd 61.74% <44.03%> (-0.02%) ⬇️
test-optimization-playwright-playwright-final-status 61.69% <44.03%> (-0.03%) ⬇️
test-optimization-playwright-playwright-impacted-tests 61.42% <44.03%> (+0.14%) ⬆️
test-optimization-playwright-playwright-reporting 61.32% <44.03%> (-0.03%) ⬇️
test-optimization-playwright-playwright-test-management 62.23% <44.03%> (-0.14%) ⬇️
test-optimization-playwright-playwright-test-span 61.46% <44.03%> (-0.08%) ⬇️
test-optimization-selenium 60.69% <71.55%> (-0.15%) ⬇️
test-optimization-testopt 59.19% <77.06%> (+0.12%) ⬆️
test-optimization-vitest 71.57% <71.55%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@BridgeAR
BridgeAR marked this pull request as ready for review July 7, 2026 22:08
@BridgeAR
BridgeAR requested review from a team as code owners July 7, 2026 22:08
@BridgeAR
BridgeAR requested review from bojbrook and khanayan123 and removed request for a team July 7, 2026 22:08

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 912851ae11

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread packages/dd-trace/src/opentracing/propagation/text_map.js Outdated
Comment thread index.d.ts Outdated
@BridgeAR
BridgeAR marked this pull request as draft July 8, 2026 08:50
BridgeAR added a commit that referenced this pull request Jul 8, 2026
Tracer.inject now reports whether it wrote context into the carrier, and
index.d.ts was updated to return boolean. index.d.v5.ts still declared
void. inject exists on v5 and this return is not v6-only, so the v5
release type swap (scripts/release/swap-v5-types.js) would ship a stale
void declaration and hide the return value from v5 TypeScript users even
though the runtime provides it. Mirror the signature and JSDoc so the two
public surfaces diverge only by the intended v6 cleanups.

Refs: #9183 (comment)
BridgeAR added a commit that referenced this pull request Jul 8, 2026
The prior commit widened Tracer.inject's declared return from void to
boolean across the public surface. That is a breaking TypeScript change:
under strict mode `const x: void = tracer.inject(...)` and returning
`tracer.inject(...)` from a void-typed function stop compiling. It must
not ship on a shipping major.

The runtime already returns the boolean and the internal AWS SDK
producers consume it through the unwrapped DatadogTracer (a plugin's
`this.tracer` resolves to `_tracer._tracer`, and the DSM checkpointer
holds that inner tracer directly), never through the public proxy. So
the boolean stays an internal signal regardless of what the public
`.d.ts` promises.

Freeze the contract per major:

1. index.d.v5.ts and the new index.d.v6.ts declare `inject(): void` —
   the current shipping majors keep their contract.
2. index.d.ts (master is 7.0.0-pre) declares `inject(): boolean` as the
   next-major public API, documented in MIGRATING.md under 6.0 to 7.0.

Generalize the release type swap: swap-v5-types.js only handled v5, so a
v6 release would publish the v7 index.d.ts. Rename it to
swap-legacy-types.js and pick index.d.v<major>.ts by the release major;
the newest major has no frozen file and is a no-op. Adding a future
frozen surface is now a single file drop. The .v<major>.ts files stay
out of the npm `files` array by design — the swap copies the chosen one
over index.d.ts at pack time, so only index.d.ts ships.

Refs: #9183 (comment)
BridgeAR added 8 commits July 10, 2026 20:42
When a Kinesis record exceeds the 1 MiB cap, injectToMessage bailed after
setDSMCheckpoint had already run: it recorded a produce checkpoint, advanced the
pathway context, and tagged pathway.hash on the span, yet params.Data was never
written so nothing shipped on the wire. The producer left a dangling pathway edge
no consumer completes and a span tagged with a hash for a message it never sent.

Gate on the trace-context payload before setDSMCheckpoint so a record we cannot
ship records no checkpoint, advances no pathway, and tags no hash. The bounded
dd-pathway-ctx-base64 bytes are added after the gate; they cannot realistically
tip a record that already cleared 1 MiB with room for them.

EventBridge shares this checkpoint-before-gate pattern and is tracked separately
in #8476; this change is Kinesis-only.
The kinesis, sqs, and sns producers decided whether to attach `_datadog`
by re-scanning the freshly built `ddInfo` carrier with an `isEmpty`
for-in helper. A local `injected` flag set at each population site
carries the same fact without the per-send rescan or the shared helper,
and matches `DsmPathwayCodec.encode`'s own `!dataStreamsContext.hash`
guard on the sns path so an empty pathway context no longer counts as
injected.

Replace the faked kinesis inject-to-message spec — an
`Object.create(Kinesis.prototype)` instance with a stubbed tracer — with
a real agent-loaded regression in kinesis.dsm.spec.js: a record that
only clears the 1 MiB cap before the trace context is attached must
record no produce checkpoint, asserted through the `putRecord` span
carrying no `pathway.hash`. Drop the sqs unit case that asserted a
carrier state (`injectTraceContext` true, `inject` a no-op) no real
caller reaches.
`Tracer.inject` and every propagator now report whether they wrote any
context, so the AWS SDK messaging producers stop attaching an empty
`_datadog` message attribute when nothing was injected. Previously they
assumed a write happened whenever injection was attempted, which shipped an
empty carrier under `DD_TRACE_PROPAGATION_STYLE_INJECT=none` and whenever a
disabled-APM-tracing (standalone ASM) trace was stripped after injection.

The standalone ASM strip runs on the `dd-trace:span:inject` channel after the
propagator writes, so the text_map propagator re-checks the `_dd.p.ts` trace
tag when APM tracing is disabled to keep the returned value truthful without
rescanning the carrier.
`Tracer.inject` now reports whether it wrote context into the carrier, and the
AWS SDK producers read that instead of rescanning with `isEmpty`. Four
expectations still modelled the old `undefined` return and failed in CI:

1. sqs-inject-to-message: stubs mutate the carrier but returned `undefined`, so
   the producer dropped `_datadog`; return `true` from the writing stubs and
   `false` from the no-op default.
2. dd-trace-api: `inject` forwards its return through the public API, now
   `false` for the empty test args; assert `false`.
3. pathway: type the no-hash `encode` argument concretely to clear
   jsdoc/reject-any-type.
4. kinesis.dsm: `assert.match(resource, /^putRecord/)` clears
   eslint-prefer-assert-match.
The size gate ran on the trace-context payload before DsmPathwayCodec.encode
appended the fixed-size dd-pathway-ctx-base64 field. A record sitting within 55
bytes of the 1 MiB cap cleared the gate, recorded a checkpoint, and then wrote an
over-cap params.Data that Kinesis rejects, failing the user's putRecord. Reserve
the pathway field's bytes in the gate so a record that only fits without it
records no checkpoint and never ships over the cap.
Tracer.inject now reports whether it wrote context into the carrier, and
index.d.ts was updated to return boolean. index.d.v5.ts still declared
void. inject exists on v5 and this return is not v6-only, so the v5
release type swap (scripts/release/swap-v5-types.js) would ship a stale
void declaration and hide the return value from v5 TypeScript users even
though the runtime provides it. Mirror the signature and JSDoc so the two
public surfaces diverge only by the intended v6 cleanups.

Refs: #9183 (comment)
The prior commit widened Tracer.inject's declared return from void to
boolean across the public surface. That is a breaking TypeScript change:
under strict mode `const x: void = tracer.inject(...)` and returning
`tracer.inject(...)` from a void-typed function stop compiling. It must
not ship on a shipping major.

The runtime already returns the boolean and the internal AWS SDK
producers consume it through the unwrapped DatadogTracer (a plugin's
`this.tracer` resolves to `_tracer._tracer`, and the DSM checkpointer
holds that inner tracer directly), never through the public proxy. So
the boolean stays an internal signal regardless of what the public
`.d.ts` promises.

Freeze the contract per major:

1. index.d.v5.ts and the new index.d.v6.ts declare `inject(): void` —
   the current shipping majors keep their contract.
2. index.d.ts (master is 7.0.0-pre) declares `inject(): boolean` as the
   next-major public API, documented in MIGRATING.md under 6.0 to 7.0.

Generalize the release type swap: swap-v5-types.js only handled v5, so a
v6 release would publish the v7 index.d.ts. Rename it to
swap-legacy-types.js and pick index.d.v<major>.ts by the release major;
the newest major has no frozen file and is a no-op. Adding a future
frozen surface is now a single file drop. The .v<major>.ts files stay
out of the npm `files` array by design — the swap copies the chosen one
over index.d.ts at pack time, so only index.d.ts ships.

Refs: #9183 (comment)
Standalone ASM can remove trace headers while leaving baggage in the carrier. A boolean result reported false for that non-empty carrier, causing AWS messaging producers to drop valid context. Return the carrier internally after its first write while keeping the public tracer API void at the proxy boundary.

Kinesis limits cover Data plus PartitionKey. Include both before recording a DSM checkpoint and allow records at the exact default limit.

No-write injection measured 11.96-12.21 ns eager vs 6.92-6.98 ns lazy; one write in eight measured 29.30-29.46 ns vs 26.09-26.66 ns on Node 24.18.0 / V8 13.6.233.17, 5M iterations x 7 trials with best and worst dropped.
@BridgeAR
BridgeAR force-pushed the BridgeAR/2026-07-02-kinesis-dsm-checkpoint-size-gate branch from 25e657b to 9d7589d Compare July 10, 2026 20:12
@BridgeAR
BridgeAR marked this pull request as ready for review July 13, 2026 12:36
@tlhunter
tlhunter merged commit 026451e into master Jul 14, 2026
714 of 715 checks passed
@tlhunter
tlhunter deleted the BridgeAR/2026-07-02-kinesis-dsm-checkpoint-size-gate branch July 14, 2026 19:37
This was referenced Jul 15, 2026
juan-fernandez pushed a commit that referenced this pull request Jul 16, 2026
…9183)

* fix(aws-sdk): record kinesis DSM checkpoint only when context ships

When a Kinesis record exceeds the 1 MiB cap, injectToMessage bailed after
setDSMCheckpoint had already run: it recorded a produce checkpoint, advanced the
pathway context, and tagged pathway.hash on the span, yet params.Data was never
written so nothing shipped on the wire. The producer left a dangling pathway edge
no consumer completes and a span tagged with a hash for a message it never sent.

Gate on the trace-context payload before setDSMCheckpoint so a record we cannot
ship records no checkpoint, advances no pathway, and tags no hash. The bounded
dd-pathway-ctx-base64 bytes are added after the gate; they cannot realistically
tip a record that already cleared 1 MiB with room for them.

EventBridge shares this checkpoint-before-gate pattern and is tracked separately
in #8476; this change is Kinesis-only.

* refactor(aws-sdk): track message injection with a boolean, drop isEmpty

The kinesis, sqs, and sns producers decided whether to attach `_datadog`
by re-scanning the freshly built `ddInfo` carrier with an `isEmpty`
for-in helper. A local `injected` flag set at each population site
carries the same fact without the per-send rescan or the shared helper,
and matches `DsmPathwayCodec.encode`'s own `!dataStreamsContext.hash`
guard on the sns path so an empty pathway context no longer counts as
injected.

Replace the faked kinesis inject-to-message spec — an
`Object.create(Kinesis.prototype)` instance with a stubbed tracer — with
a real agent-loaded regression in kinesis.dsm.spec.js: a record that
only clears the 1 MiB cap before the trace context is attached must
record no produce checkpoint, asserted through the `putRecord` span
carrying no `pathway.hash`. Drop the sqs unit case that asserted a
carrier state (`injectTraceContext` true, `inject` a no-op) no real
caller reaches.

* refactor(core): return whether inject wrote anything into the carrier

`Tracer.inject` and every propagator now report whether they wrote any
context, so the AWS SDK messaging producers stop attaching an empty
`_datadog` message attribute when nothing was injected. Previously they
assumed a write happened whenever injection was attempted, which shipped an
empty carrier under `DD_TRACE_PROPAGATION_STYLE_INJECT=none` and whenever a
disabled-APM-tracing (standalone ASM) trace was stripped after injection.

The standalone ASM strip runs on the `dd-trace:span:inject` channel after the
propagator writes, so the text_map propagator re-checks the `_dd.p.ts` trace
tag when APM tracing is disabled to keep the returned value truthful without
rescanning the carrier.

* test: align inject test doubles with the boolean carrier-write return

`Tracer.inject` now reports whether it wrote context into the carrier, and the
AWS SDK producers read that instead of rescanning with `isEmpty`. Four
expectations still modelled the old `undefined` return and failed in CI:

1. sqs-inject-to-message: stubs mutate the carrier but returned `undefined`, so
   the producer dropped `_datadog`; return `true` from the writing stubs and
   `false` from the no-op default.
2. dd-trace-api: `inject` forwards its return through the public API, now
   `false` for the empty test args; assert `false`.
3. pathway: type the no-hash `encode` argument concretely to clear
   jsdoc/reject-any-type.
4. kinesis.dsm: `assert.match(resource, /^putRecord/)` clears
   eslint-prefer-assert-match.

* fix(aws-sdk): reserve the DSM pathway field before the kinesis size gate

The size gate ran on the trace-context payload before DsmPathwayCodec.encode
appended the fixed-size dd-pathway-ctx-base64 field. A record sitting within 55
bytes of the 1 MiB cap cleared the gate, recorded a checkpoint, and then wrote an
over-cap params.Data that Kinesis rejects, failing the user's putRecord. Reserve
the pathway field's bytes in the gate so a record that only fits without it
records no checkpoint and never ships over the cap.

* fix(types): mirror inject boolean return in the v5 type surface

Tracer.inject now reports whether it wrote context into the carrier, and
index.d.ts was updated to return boolean. index.d.v5.ts still declared
void. inject exists on v5 and this return is not v6-only, so the v5
release type swap (scripts/release/swap-v5-types.js) would ship a stale
void declaration and hide the return value from v5 TypeScript users even
though the runtime provides it. Mirror the signature and JSDoc so the two
public surfaces diverge only by the intended v6 cleanups.

Refs: #9183 (comment)

* fix(types): keep public inject void on v5/v6, expose boolean on v7

The prior commit widened Tracer.inject's declared return from void to
boolean across the public surface. That is a breaking TypeScript change:
under strict mode `const x: void = tracer.inject(...)` and returning
`tracer.inject(...)` from a void-typed function stop compiling. It must
not ship on a shipping major.

The runtime already returns the boolean and the internal AWS SDK
producers consume it through the unwrapped DatadogTracer (a plugin's
`this.tracer` resolves to `_tracer._tracer`, and the DSM checkpointer
holds that inner tracer directly), never through the public proxy. So
the boolean stays an internal signal regardless of what the public
`.d.ts` promises.

Freeze the contract per major:

1. index.d.v5.ts and the new index.d.v6.ts declare `inject(): void` —
   the current shipping majors keep their contract.
2. index.d.ts (master is 7.0.0-pre) declares `inject(): boolean` as the
   next-major public API, documented in MIGRATING.md under 6.0 to 7.0.

Generalize the release type swap: swap-v5-types.js only handled v5, so a
v6 release would publish the v7 index.d.ts. Rename it to
swap-legacy-types.js and pick index.d.v<major>.ts by the release major;
the newest major has no frozen file and is a no-op. Adding a future
frozen surface is now a single file drop. The .v<major>.ts files stay
out of the npm `files` array by design — the swap copies the chosen one
over index.d.ts at pack time, so only index.d.ts ships.

Refs: #9183 (comment)

* fix(propagation): return lazily created carriers

Standalone ASM can remove trace headers while leaving baggage in the carrier. A boolean result reported false for that non-empty carrier, causing AWS messaging producers to drop valid context. Return the carrier internally after its first write while keeping the public tracer API void at the proxy boundary.

Kinesis limits cover Data plus PartitionKey. Include both before recording a DSM checkpoint and allow records at the exact default limit.

No-write injection measured 11.96-12.21 ns eager vs 6.92-6.98 ns lazy; one write in eight measured 29.30-29.46 ns vs 26.09-26.66 ns on Node 24.18.0 / V8 13.6.233.17, 5M iterations x 7 trials with best and worst dropped.
juan-fernandez pushed a commit that referenced this pull request Jul 16, 2026
…9183)

* fix(aws-sdk): record kinesis DSM checkpoint only when context ships

When a Kinesis record exceeds the 1 MiB cap, injectToMessage bailed after
setDSMCheckpoint had already run: it recorded a produce checkpoint, advanced the
pathway context, and tagged pathway.hash on the span, yet params.Data was never
written so nothing shipped on the wire. The producer left a dangling pathway edge
no consumer completes and a span tagged with a hash for a message it never sent.

Gate on the trace-context payload before setDSMCheckpoint so a record we cannot
ship records no checkpoint, advances no pathway, and tags no hash. The bounded
dd-pathway-ctx-base64 bytes are added after the gate; they cannot realistically
tip a record that already cleared 1 MiB with room for them.

EventBridge shares this checkpoint-before-gate pattern and is tracked separately
in #8476; this change is Kinesis-only.

* refactor(aws-sdk): track message injection with a boolean, drop isEmpty

The kinesis, sqs, and sns producers decided whether to attach `_datadog`
by re-scanning the freshly built `ddInfo` carrier with an `isEmpty`
for-in helper. A local `injected` flag set at each population site
carries the same fact without the per-send rescan or the shared helper,
and matches `DsmPathwayCodec.encode`'s own `!dataStreamsContext.hash`
guard on the sns path so an empty pathway context no longer counts as
injected.

Replace the faked kinesis inject-to-message spec — an
`Object.create(Kinesis.prototype)` instance with a stubbed tracer — with
a real agent-loaded regression in kinesis.dsm.spec.js: a record that
only clears the 1 MiB cap before the trace context is attached must
record no produce checkpoint, asserted through the `putRecord` span
carrying no `pathway.hash`. Drop the sqs unit case that asserted a
carrier state (`injectTraceContext` true, `inject` a no-op) no real
caller reaches.

* refactor(core): return whether inject wrote anything into the carrier

`Tracer.inject` and every propagator now report whether they wrote any
context, so the AWS SDK messaging producers stop attaching an empty
`_datadog` message attribute when nothing was injected. Previously they
assumed a write happened whenever injection was attempted, which shipped an
empty carrier under `DD_TRACE_PROPAGATION_STYLE_INJECT=none` and whenever a
disabled-APM-tracing (standalone ASM) trace was stripped after injection.

The standalone ASM strip runs on the `dd-trace:span:inject` channel after the
propagator writes, so the text_map propagator re-checks the `_dd.p.ts` trace
tag when APM tracing is disabled to keep the returned value truthful without
rescanning the carrier.

* test: align inject test doubles with the boolean carrier-write return

`Tracer.inject` now reports whether it wrote context into the carrier, and the
AWS SDK producers read that instead of rescanning with `isEmpty`. Four
expectations still modelled the old `undefined` return and failed in CI:

1. sqs-inject-to-message: stubs mutate the carrier but returned `undefined`, so
   the producer dropped `_datadog`; return `true` from the writing stubs and
   `false` from the no-op default.
2. dd-trace-api: `inject` forwards its return through the public API, now
   `false` for the empty test args; assert `false`.
3. pathway: type the no-hash `encode` argument concretely to clear
   jsdoc/reject-any-type.
4. kinesis.dsm: `assert.match(resource, /^putRecord/)` clears
   eslint-prefer-assert-match.

* fix(aws-sdk): reserve the DSM pathway field before the kinesis size gate

The size gate ran on the trace-context payload before DsmPathwayCodec.encode
appended the fixed-size dd-pathway-ctx-base64 field. A record sitting within 55
bytes of the 1 MiB cap cleared the gate, recorded a checkpoint, and then wrote an
over-cap params.Data that Kinesis rejects, failing the user's putRecord. Reserve
the pathway field's bytes in the gate so a record that only fits without it
records no checkpoint and never ships over the cap.

* fix(types): mirror inject boolean return in the v5 type surface

Tracer.inject now reports whether it wrote context into the carrier, and
index.d.ts was updated to return boolean. index.d.v5.ts still declared
void. inject exists on v5 and this return is not v6-only, so the v5
release type swap (scripts/release/swap-v5-types.js) would ship a stale
void declaration and hide the return value from v5 TypeScript users even
though the runtime provides it. Mirror the signature and JSDoc so the two
public surfaces diverge only by the intended v6 cleanups.

Refs: #9183 (comment)

* fix(types): keep public inject void on v5/v6, expose boolean on v7

The prior commit widened Tracer.inject's declared return from void to
boolean across the public surface. That is a breaking TypeScript change:
under strict mode `const x: void = tracer.inject(...)` and returning
`tracer.inject(...)` from a void-typed function stop compiling. It must
not ship on a shipping major.

The runtime already returns the boolean and the internal AWS SDK
producers consume it through the unwrapped DatadogTracer (a plugin's
`this.tracer` resolves to `_tracer._tracer`, and the DSM checkpointer
holds that inner tracer directly), never through the public proxy. So
the boolean stays an internal signal regardless of what the public
`.d.ts` promises.

Freeze the contract per major:

1. index.d.v5.ts and the new index.d.v6.ts declare `inject(): void` —
   the current shipping majors keep their contract.
2. index.d.ts (master is 7.0.0-pre) declares `inject(): boolean` as the
   next-major public API, documented in MIGRATING.md under 6.0 to 7.0.

Generalize the release type swap: swap-v5-types.js only handled v5, so a
v6 release would publish the v7 index.d.ts. Rename it to
swap-legacy-types.js and pick index.d.v<major>.ts by the release major;
the newest major has no frozen file and is a no-op. Adding a future
frozen surface is now a single file drop. The .v<major>.ts files stay
out of the npm `files` array by design — the swap copies the chosen one
over index.d.ts at pack time, so only index.d.ts ships.

Refs: #9183 (comment)

* fix(propagation): return lazily created carriers

Standalone ASM can remove trace headers while leaving baggage in the carrier. A boolean result reported false for that non-empty carrier, causing AWS messaging producers to drop valid context. Return the carrier internally after its first write while keeping the public tracer API void at the proxy boundary.

Kinesis limits cover Data plus PartitionKey. Include both before recording a DSM checkpoint and allow records at the exact default limit.

No-write injection measured 11.96-12.21 ns eager vs 6.92-6.98 ns lazy; one write in eight measured 29.30-29.46 ns vs 26.09-26.66 ns on Node 24.18.0 / V8 13.6.233.17, 5M iterations x 7 trials with best and worst dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants