test(kafkajs): stop pinning the produce offset in the sendBatch test#9074
Conversation
The "should emit one kafka.produce span per topicMessages entry" test hard-coded kafka.messages.offsets to start_offset "0". Kafka produce is at-least-once: a transient NOT_LEADER_FOR_PARTITION right after topic creation makes kafkajs retry and advance the broker-assigned base offset past 0, so the span faithfully reports a non-zero offset (CI observed "1") and the assertion never matched before the timeout. The expected offsets are read back from the sendBatch result instead, which still pins the per-topic isolation the test was written for. Each topicMessages entry is its own root span, so the two spans are separate traces the agent may deliver in a single payload in any order; the span lookup now scans every trace rather than only traces[0].
Overall package sizeSelf size: 6.35 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 0dde63c | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-06-25 23:21:15 Comparing candidate commit 0dde63c in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2269 metrics, 14 unstable metrics.
|
…9074) The "should emit one kafka.produce span per topicMessages entry" test hard-coded kafka.messages.offsets to start_offset "0". Kafka produce is at-least-once: a transient NOT_LEADER_FOR_PARTITION right after topic creation makes kafkajs retry and advance the broker-assigned base offset past 0, so the span faithfully reports a non-zero offset (CI observed "1") and the assertion never matched before the timeout. The expected offsets are read back from the sendBatch result instead, which still pins the per-topic isolation the test was written for. Each topicMessages entry is its own root span, so the two spans are separate traces the agent may deliver in a single payload in any order; the span lookup now scans every trace rather than only traces[0].
…9074) The "should emit one kafka.produce span per topicMessages entry" test hard-coded kafka.messages.offsets to start_offset "0". Kafka produce is at-least-once: a transient NOT_LEADER_FOR_PARTITION right after topic creation makes kafkajs retry and advance the broker-assigned base offset past 0, so the span faithfully reports a non-zero offset (CI observed "1") and the assertion never matched before the timeout. The expected offsets are read back from the sendBatch result instead, which still pins the per-topic isolation the test was written for. Each topicMessages entry is its own root span, so the two spans are separate traces the agent may deliver in a single payload in any order; the span lookup now scans every trace rather than only traces[0].
2 participants
Summary
Fixes a flake in
Plugin > kafkajs > producer (sendBatch) > should emit one kafka.produce span per topicMessages entry(surfaced on thekafkajs (oldest)job).The test hard-coded
kafka.messages.offsetstostart_offset: "0". Kafka produce is at-least-once: a transientNOT_LEADER_FOR_PARTITIONright after topic creation makes kafkajs retry and advance the broker-assigned base offset past 0. The span correctly reports the broker's real offset (CI observed"1"), so the per-topicassertSomeTracesnever matched and timed out. The hard-coded0was the only defect — the expected offsets are now read back from thesendBatchresult, so the assertion tracks the broker while still pinning per-topic isolation, the property the test exists for. That isolation is also covered deterministically (including non-zero offsets) bycommit.spec.js.A second, independent fragility: each
topicMessagesentry is its own root span, so the two spans are separate traces the agent may deliver in one payload in any order. The lookup now scans every trace instead of onlytraces[0].Test plan
kafkajsplugin CI matrix (PLUGINS=kafkajs SERVICES=kafka) green across the supported kafkajs versions.Local:
commit.spec.jspasses andeslintis clean; the integration suite needs a live Kafka broker, unavailable in this environment.Refs: #8403