[issues/14372] support ListIterator for Kafka consumer client (2)#14757
Merged
trask merged 12 commits intoopen-telemetry:mainfrom Sep 26, 2025
Merged
[issues/14372] support ListIterator for Kafka consumer client (2)#14757trask merged 12 commits intoopen-telemetry:mainfrom
trask merged 12 commits intoopen-telemetry:mainfrom
Conversation
Contributor
|
🔧 The result from spotlessApply was committed to the PR branch. |
laurit
approved these changes
Sep 25, 2025
Contributor
Author
|
Thank you @laurit for the journey 🙂 |
Contributor
Author
Test result using Spark |
Contributor
|
Thank you for your contribution @patpatpat123! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
( I accidentaly closed #14707, my bad)
Hello team,
This PR is to address issue #14372
Problem it is trying to solve:
As of today, Spark Structured Streaming Kafka Consumer, upon consuming a message with a valid W3C header with trace, is not continuing, nor propagating the trace on the Spark side.
Root cause analysis:
It seems Spark Structured Streaming Kafka Consumer is relying on
listIteratorC.f. https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/consumer/KafkaDataConsumer.scala#L593As of now, Kafka instrumentation does not support
listIterator.Proposed solution
This solution will add
TracingListIterator<K, V> implements ListIterator<ConsumerRecord<K, V>>(TracingListIterator.java) which will allow the handling oflistIterator(instrumentation/kafka/kafka-clients/kafka-clients-common-0.11/library/src/main/java/io/opentelemetry/instrumentation/kafkaclients/common/v0_11/internal/TracingList.java )Thanks
Many thanks Lauri, Trask, and Jay for the help provided