Skip to content

kafka.consumer.* metrics with NaN values and no unit #9498

Description

@mviitane

Describe the bug

I’m running OpenTelemetry demo application with the latest v1.30.0 java agent. My metrics backend complains about receiving a lot of NaN value metrics.

It looks like the Fraud Detection service (Kafka consumer in the demo app) creates these NaN value metrics. Also, these metrics are missing the unit.

Steps to reproduce

  • Download OpenTelemetry demo application (use main/latest, not the latest released version, to have the latest java agent)
  • Configure detailed logging exporter in the Collector to access the detailed metric printouts
exporters:
  logging/detailed:
    verbosity: detailed
  • Start the demo app
  • After around 2 min, read the Collector logs with "NaN" values
    $ docker compose logs otelcol | grep -B 11 NaN

Expected behavior

I'd expect all the exported metrics to have a value and a unit. NaN value metrics shouldn't be propagated.

Actual behavior

Lots of metrics with NaN values and no unit:

otel-col  | Metric #72
otel-col  | Descriptor:
otel-col  |      -> Name: kafka.consumer.partition_assigned_latency_max
otel-col  |      -> Description: The max time taken for a partition-assigned rebalance listener callback
otel-col  |      -> Unit: 
otel-col  |      -> DataType: Gauge
otel-col  | NumberDataPoints #0
otel-col  | Data point attributes:
otel-col  |      -> client-id: Str(consumer-frauddetectionservice-1)
otel-col  | StartTimestamp: 2023-09-19 06:14:58.534111254 +0000 UTC
otel-col  | Timestamp: 2023-09-19 06:15:58.53179417 +0000 UTC
otel-col  | Value: NaN
--
otel-col  | Metric #77
otel-col  | Descriptor:
otel-col  |      -> Name: kafka.consumer.join_time_avg
otel-col  |      -> Description: The average time taken for a group rejoin
otel-col  |      -> Unit: 
otel-col  |      -> DataType: Gauge
otel-col  | NumberDataPoints #0
otel-col  | Data point attributes:
otel-col  |      -> client-id: Str(consumer-frauddetectionservice-1)
otel-col  | StartTimestamp: 2023-09-19 06:14:58.534111254 +0000 UTC
otel-col  | Timestamp: 2023-09-19 06:15:58.53179417 +0000 UTC
otel-col  | Value: NaN
--
otel-col  | Metric #84
otel-col  | Descriptor:
otel-col  |      -> Name: kafka.consumer.rebalance_latency_max
otel-col  |      -> Description: The max time taken for a group to complete a successful rebalance, which may be composed of several failed re-trials until it succeeded
otel-col  |      -> Unit: 
otel-col  |      -> DataType: Gauge
otel-col  | NumberDataPoints #0
otel-col  | Data point attributes:
otel-col  |      -> client-id: Str(consumer-frauddetectionservice-1)
otel-col  | StartTimestamp: 2023-09-19 06:14:58.534111254 +0000 UTC
otel-col  | Timestamp: 2023-09-19 06:15:58.53179417 +0000 UTC
otel-col  | Value: NaN
--
otel-col  | Metric #85
otel-col  | Descriptor:
otel-col  |      -> Name: kafka.consumer.reauthentication_latency_avg
otel-col  |      -> Description: The average latency observed due to re-authentication
otel-col  |      -> Unit: 
otel-col  |      -> DataType: Gauge
otel-col  | NumberDataPoints #0
otel-col  | Data point attributes:
otel-col  |      -> client-id: Str(consumer-frauddetectionservice-1)
otel-col  | StartTimestamp: 2023-09-19 06:14:58.534111254 +0000 UTC
otel-col  | Timestamp: 2023-09-19 06:15:58.53179417 +0000 UTC
otel-col  | Value: NaN

Javaagent or library instrumentation version

v1.30.0

Environment

Docker desktop on macOS

Additional context

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#numerical-limits-handling

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds triageNew issue that requires triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions