feat(dsm): add kafka_cluster_id to segmentio/kafka-go integration by robcarlan-datadog · Pull Request #4477 · DataDog/dd-trace-go

robcarlan-datadog · 2026-02-26T17:15:24Z

Summary

Adds kafka_cluster_id support to the segmentio/kafka-go DSM integration, building on feat(dsm): add kafka_cluster_id to confluent-kafka-go #4470
Auto-fetches the Kafka cluster ID via async metadata request in WrapReader/WrapWriter
Plumbs cluster ID through DSM edge tags, offset tracking, and span tags

Tested this manually and via unit/integration tests.

Plumb Kafka cluster ID through the DSM pipeline so that offsets and checkpoints from different clusters are distinguishable. For confluent-kafka-go, the cluster ID is fetched automatically via the AdminClient API; other Kafka libraries pass empty string for now and will be wired in follow-up PRs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Add cluster ID as a span tag on produce and consume spans, add ext constant MessagingKafkaClusterID, and add integration test verifying cluster ID flows into spans, DSM pathway hashes, and backlog tags. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Auto-fetched cluster ID from the Kafka admin API is now prepended to options so that user-provided WithClusterID takes precedence. Also fix TestConsumerFunctional pathway hash to account for auto-fetched cluster ID in integration test environments. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The TestConsumerFunctional test was computing expected pathway hashes without the kafka_cluster_id tag, but the production code now includes it after auto-fetching from the broker. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

pr-commenter · 2026-02-26T17:33:20Z

Benchmarks

Benchmark execution time: 2026-03-20 04:19:21

Comparing candidate commit 28e8a60 in PR branch rob.carlan/kafka-go-cluster-id with baseline commit 6ae9763 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 216 metrics, 8 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

codecov · 2026-02-26T17:36:01Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.19%. Comparing base (5ef1f76) to head (3f8fc7b).

Additional details and impacted files

see 12 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Avoid repeated admin API calls to fetch cluster ID when creating multiple consumers/producers to the same Kafka cluster. The cache is shared between v1 and v2 packages via kafkatrace. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Remove the public WithClusterID option function which was only used internally and for testing. Set cluster ID directly on the tracer after wrapping instead. Also reduce cluster ID fetch timeout from 10s to 2s. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Move cluster ID fetching to a background goroutine so it doesn't block Consumer/Producer creation. Protect the clusterID field with a RWMutex for safe concurrent access. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Move cluster ID goroutine lifecycle into the Tracer via FetchClusterIDAsync/WaitForClusterID. Close() waits for the goroutine to prevent use-after-free of the librdkafka handle. Tests call WaitForClusterID before producing/consuming to ensure deterministic pathway hashes. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

datadog-official · 2026-03-04T16:47:11Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.35% (-0.07%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 28e8a60 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

… TrackKafkaProduceOffsetWithCluster params FetchClusterIDAsync now accepts a cancellable context so Close() can signal the fetch to stop early rather than blocking unconditionally. Also reorders TrackKafkaProduceOffsetWithCluster params to match the Kafka hierarchy: cluster, topic, partition, offset. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Replace duplicated cluster ID fetching, caching, and normalization code with the shared instrumentation/kafkaclusterid package. Use context-aware FetchAsync for cancellation support on Close. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

… ID impl Replace the kafkaclusterid.Fetcher with a simple mutex-guarded string matching the confluent-kafka-go pattern. Fix TrackKafkaCommitOffsetWithCluster argument order. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

kafka.TCP expects variadic individual addresses, not a single comma-separated string. Passing one string causes a network/address array length mismatch in kafka-go's transport.go Shuffle, panicking with index out of range. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Add require.Eventually to genIntegrationTestSpans to wait for the async cluster ID fetch to complete, matching the confluent-kafka-go test pattern. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

genesor

Looks great! I have one quick comment about the mutex.

…go-cluster-id

robcarlan-datadog · 2026-03-20T13:47:37Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-20T13:47:41Z

View all feedbacks in Devflow UI.

2026-03-20 13:47:41 UTC ℹ️ Start processing command /merge

2026-03-20 13:47:45 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 30m (p90).

2026-03-20 13:57:56 UTC ℹ️ MergeQueue: This merge request was merged

## Summary - This follows practically the same pattern as the [confluent-kafka-go](#4470) and [kafka-go](#4477) PRs. However, this integration requires the user to supply the bootstrap servers explicitly in order to obtain the cluster id. - Adds `WithBrokers` option to both IBM/sarama and Shopify/sarama integrations that fetches the Kafka cluster ID via a metadata request and includes it in DSM payloads (DSM only) - Tested manually and via unit / integration tests. Co-authored-by: genesor <[email protected]> Co-authored-by: rob.carlan <[email protected]>

robcarlan-datadog and others added 6 commits February 26, 2026 12:11

chore: update api.txt for new DSM cluster ID functions

e0ad51d

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

fix tests

354ebc1

github-actions Bot added the apm:ecosystem contrib/* related feature requests or bugs label Feb 26, 2026

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 76111c5 to 7d0d2b3 Compare February 26, 2026 17:29

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 7d0d2b3 to 4c9fc7d Compare February 26, 2026 18:38

robcarlan-datadog marked this pull request as ready for review February 26, 2026 18:46

robcarlan-datadog requested review from a team as code owners February 26, 2026 18:46

darccio approved these changes Mar 2, 2026

View reviewed changes

robcarlan-datadog and others added 3 commits March 2, 2026 12:25

remove comments

5514993

Merge branch 'main' into rob.carlan/dsm-kafka-cluster-id

388479b

robcarlan-datadog added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Mar 2, 2026

robcarlan-datadog and others added 2 commits March 4, 2026 10:32

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 68e76bf to 3ed2642 Compare March 4, 2026 16:15

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 52c3314 to 01f3ee5 Compare March 4, 2026 17:58

robcarlan-datadog force-pushed the rob.carlan/dsm-kafka-cluster-id branch from 6dcd1dc to f98f0a4 Compare March 5, 2026 21:01

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from ac5efc0 to e39a9ab Compare March 5, 2026 21:01

robcarlan-datadog force-pushed the rob.carlan/dsm-kafka-cluster-id branch from f98f0a4 to e916411 Compare March 6, 2026 17:09

robcarlan-datadog and others added 3 commits March 18, 2026 13:01

fix: remove trailing blank line in option.go

e8fc549

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from e39a9ab to 1776bdc Compare March 18, 2026 17:14

robcarlan-datadog and others added 3 commits March 18, 2026 13:26

Merge branch 'main' into rob.carlan/dsm-kafka-cluster-id

9568d67

refactor kafka-go implementation

04f2de2

Add concurrency test for cluster ID access

89413af

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from b706da3 to 89413af Compare March 18, 2026 18:38

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 0d976d2 to 783bddd Compare March 18, 2026 21:32

fix: wait for async cluster ID fetch before producing/consuming in tests

443d973

Add require.Eventually to genIntegrationTestSpans to wait for the async cluster ID fetch to complete, matching the confluent-kafka-go test pattern. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

robcarlan-datadog force-pushed the rob.carlan/kafka-go-cluster-id branch from 783bddd to 443d973 Compare March 18, 2026 21:41

robcarlan-datadog requested a review from genesor March 18, 2026 22:20

genesor reviewed Mar 18, 2026

View reviewed changes

Comment thread contrib/segmentio/kafka-go/internal/tracing/tracer.go Outdated

robcarlan-datadog added 3 commits March 19, 2026 10:38

Merge branch 'main' into rob.carlan/dsm-kafka-cluster-id

5ef1f76

use checkatommic

dd7dfae

Merge branch 'rob.carlan/dsm-kafka-cluster-id' into rob.carlan/kafka-…

3f8fc7b

…go-cluster-id

Base automatically changed from rob.carlan/dsm-kafka-cluster-id to main March 19, 2026 23:37

Merge branch 'main' into rob.carlan/kafka-go-cluster-id

28e8a60

robcarlan-datadog mentioned this pull request Mar 20, 2026

feat(dsm): add kafka_cluster_id to Sarama integrations #4560

Merged

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Mar 20, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 516cca3 into main Mar 20, 2026
92 checks passed

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Mar 20, 2026

gh-worker-dd-mergequeue-cf854d Bot deleted the rob.carlan/kafka-go-cluster-id branch March 20, 2026 13:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(dsm): add kafka_cluster_id to segmentio/kafka-go integration#4477

feat(dsm): add kafka_cluster_id to segmentio/kafka-go integration#4477
gh-worker-dd-mergequeue-cf854d[bot] merged 44 commits into
mainfrom
rob.carlan/kafka-go-cluster-id

robcarlan-datadog commented Feb 26, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Feb 26, 2026 •

edited

Loading

Explanation

More details about the CI and significant changes

Uh oh!

codecov Bot commented Feb 26, 2026 •

edited

Loading

Uh oh!

datadog-official Bot commented Mar 4, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

Uh oh!

genesor left a comment

Uh oh!

Uh oh!

robcarlan-datadog commented Mar 20, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Mar 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

robcarlan-datadog commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

pr-commenter Bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Uh oh!

codecov Bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

datadog-official Bot commented Mar 4, 2026 • edited by datadog-prod-us1-5 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

genesor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

robcarlan-datadog commented Mar 20, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

robcarlan-datadog commented Feb 26, 2026 •

edited

Loading

pr-commenter Bot commented Feb 26, 2026 •

edited

Loading

codecov Bot commented Feb 26, 2026 •

edited

Loading

datadog-official Bot commented Mar 4, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

gh-worker-devflow-routing-ef8351 Bot commented Mar 20, 2026 •

edited

Loading