feat(kafkajs): instrument producer.sendBatch#8403
Merged
Merged
Conversation
Contributor
Overall package sizeSelf size: 5.86 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
✨ Fix all issues with BitsAI or with Cursor
|
BenchmarksBenchmark execution time: 2026-05-23 07:17:45 Comparing candidate commit a3841e8 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1495 metrics, 98 unstable metrics. |
BridgeAR
force-pushed
the
BridgeAR/2026-05-09-kafka-skip-injection-pre-v3
branch
from
May 12, 2026 14:37
681b6c4 to
9792e32
Compare
Base automatically changed from
BridgeAR/2026-05-09-kafka-skip-injection-pre-v3
to
master
May 19, 2026 00:10
BridgeAR
force-pushed
the
BridgeAR/2026-05-10-kafkajs-send-batch
branch
2 times, most recently
from
May 19, 2026 23:50
9858515 to
97dc21c
Compare
Member
Author
|
This is blocked on a fix to instanbuljs for generating the coverage properly. See istanbuljs/istanbuljs#845 |
Collaborator
|
A new linting rule just landed in |
This adds tracing, header injection, and DSM checkpoints for the kafkajs producer.sendBatch API, which was previously uninstrumented: calls produced no spans and propagated no trace or DSM context. One kafka.produce span per topicMessages[] entry matches what producer.send already emits for the single-topic case and mirrors how kafkajs implements send internally (producer/messageProducer.js delegates send to a single-entry sendBatch). Per-entry granularity keeps DSM checkpoints correct on multi-topic batches. The broker response is one aggregated array covering every topic, so the plugin's finish filters by ctx.topic before tagging kafka.messages.offsets; otherwise each per-topic span would carry every other topic's offsets. Fixes: #1711 Refs: #8468
The integration spec in `packages/datadog-plugin-kafkajs/test/index.spec.js` needs Docker Kafka and always connects the producer before sending, so the slow-path branch through `cluster.refreshMetadataIfNecessary().then(...)` and its `refreshMetadataIfNecessary`-missing fallback never ran outside that one CI flag. This adds an instrumentation-level spec that drives the boundary with a faked cluster, pinning the resolve / reject / missing arms of the slow path, the proactive and reactive `refreshHeaderSupport` disable, and the `producerStartCh.hasSubscribers` fast skip without spinning a broker. Refs: #8390
The instrumentation-level spec at `packages/datadog-instrumentations/test/kafkajs.spec.js` is matched by `test:instrumentations`'s `PLUGINS=kafkajs` glob, but no workflow runs it. Without a job, codecov sees no session for the producer-boundary slow path or the `refreshHeaderSupport` disable arms.
The previous instrumentation-level spec only exercised `producer.send`. The integration spec needs Docker Kafka, so the sendBatch boundary's empty-batch fast skip, non-array messages else arm, sync-throw catch, and `KafkaJSProtocolError UNKNOWN` disable branch stay uncovered without faked-cluster unit tests.
Two of the new sendBatch cases duplicate branches the existing spec already pins: 1. The `empty topicMessages` case hit the same length-0 fast skip as the `missing topicMessages` case from a different ternary arm whose true side every multi-entry test already exercises. 2. The `clusterId after refreshMetadataIfNecessary resolves` case re-ran `withClusterId`'s resolve arm through sendBatch when the send-side slow-path test already covers all three withClusterId arms and every fast-path sendBatch test wires the boundary through L116.
BridgeAR
force-pushed
the
BridgeAR/2026-05-10-kafkajs-send-batch
branch
from
May 22, 2026 13:47
cec38d6 to
774b242
Compare
tlhunter
approved these changes
May 26, 2026
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The kafkajs
producer.sendBatchAPI was uninstrumented — calls produced no spans, propagated no trace context, and emitted no DSM checkpoints. This adds tracing, header injection, and DSM checkpoints, modeled as onekafka.producespan pertopicMessages[]entry (mirrors whatproducer.sendalready emits for the single-topic case and how kafkajs decomposessendintosendBatchinternally).Fixes: #1711