fix(datastreams): flush on write when flushInterval is 0#9120
Merged
Conversation
With `flushInterval: 0` (the test-agent config and the AWS Lambda config) the processor armed `setInterval(onInterval, 0)`, which fires on every event-loop tick instead of when a checkpoint is recorded. A tick landing in the window between the test agent tearing its listener down and bringing it back up posts to a dead port; the bucket is cleared on serialize, so that single payload is lost. A producer-only DSM test waiting on exactly one payload then times out. Honor `flushInterval === 0` as the flush-on-write sentinel the agent and agentless trace exporters already use: skip the timer and push each checkpoint, offset, and transaction the moment it is recorded, while the writer URL is live. Refs: https://github.com/DataDog/dd-trace-js/actions/runs/28302648699/job/83853526808
Contributor
Overall package sizeSelf size: 6.4 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 8717cf0 | Docs | Datadog PR Page | Give us feedback! |
BridgeAR
marked this pull request as ready for review
June 27, 2026 22:24
BenchmarksBenchmark execution time: 2026-06-27 22:25:02 Comparing candidate commit 8717cf0 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2259 metrics, 27 unstable metrics.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9120 +/- ##
=======================================
Coverage 93.69% 93.69%
=======================================
Files 889 889
Lines 50856 50860 +4
Branches 11830 11833 +3
=======================================
+ Hits 47647 47651 +4
Misses 3209 3209
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
crysmags
approved these changes
Jul 2, 2026
Merged
dd-octo-sts Bot
pushed a commit
that referenced
this pull request
Jul 2, 2026
With `flushInterval: 0` (the test-agent config and the AWS Lambda config) the processor armed `setInterval(onInterval, 0)`, which fires on every event-loop tick instead of when a checkpoint is recorded. A tick landing in the window between the test agent tearing its listener down and bringing it back up posts to a dead port; the bucket is cleared on serialize, so that single payload is lost. A producer-only DSM test waiting on exactly one payload then times out. Honor `flushInterval === 0` as the flush-on-write sentinel the agent and agentless trace exporters already use: skip the timer and push each checkpoint, offset, and transaction the moment it is recorded, while the writer URL is live.
Merged
rochdev
pushed a commit
that referenced
this pull request
Jul 2, 2026
With `flushInterval: 0` (the test-agent config and the AWS Lambda config) the processor armed `setInterval(onInterval, 0)`, which fires on every event-loop tick instead of when a checkpoint is recorded. A tick landing in the window between the test agent tearing its listener down and bringing it back up posts to a dead port; the bucket is cleared on serialize, so that single payload is lost. A producer-only DSM test waiting on exactly one payload then times out. Honor `flushInterval === 0` as the flush-on-write sentinel the agent and agentless trace exporters already use: skip the timer and push each checkpoint, offset, and transaction the moment it is recorded, while the writer URL is live.
rochdev
pushed a commit
that referenced
this pull request
Jul 2, 2026
With `flushInterval: 0` (the test-agent config and the AWS Lambda config) the processor armed `setInterval(onInterval, 0)`, which fires on every event-loop tick instead of when a checkpoint is recorded. A tick landing in the window between the test agent tearing its listener down and bringing it back up posts to a dead port; the bucket is cleared on serialize, so that single payload is lost. A producer-only DSM test waiting on exactly one payload then times out. Honor `flushInterval === 0` as the flush-on-write sentinel the agent and agentless trace exporters already use: skip the timer and push each checkpoint, offset, and transaction the moment it is recorded, while the writer URL is live.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The amqplib DSM spec (
Should emit DSM stats to the agent when sending a message on an unnamed exchange) flakes with a timeout because theDataStreamsProcessoris the onlyflushIntervalconsumer that doesn't honorflushInterval === 0as a flush-on-write sentinel.Under the test-agent config (
flushInterval: 0, the same value the AWS Lambda path uses), the processor armedsetInterval(onInterval, 0). That timer fires on every event-loop tick, decoupled from when a checkpoint is recorded. A tick that lands in the window where the test agent has torn its HTTP listener down and not yet brought it back up posts to a dead port (ECONNREFUSED); buckets are cleared on serialize, so that single payload is gone. A producer-only DSM test waiting on exactly one payload then times out.The fix aligns the processor with the agent and agentless trace exporters, which already branch on
flushInterval === 0: skip the timer and flush each checkpoint, offset, and transaction the moment it is recorded, while the writer URL is known live.Test plan
packages/dd-trace/test/datastreams/flush_on_write.spec.jsis new: it pins both branches —flushInterval: 0arms no timer and flushes synchronously onrecordCheckpoint/recordOffset/trackTransaction; a non-zero interval arms the timer and does not flush on record. It fails on master (the timer is armed, nothing flushes on write) and passes with the fix.packages/dd-trace/test/datastreams/*.spec.jspass unchanged.Refs: https://github.com/DataDog/dd-trace-js/actions/runs/28302648699/job/83853526808