Skip to content

fix(trace stats): Configure span kind and peer tag for agent-side stats#1189

Merged
lucaspimentel merged 11 commits into
mainfrom
lpimentel/fix-stats-span-kind-peer-tags
Jun 15, 2026
Merged

fix(trace stats): Configure span kind and peer tag for agent-side stats#1189
lucaspimentel merged 11 commits into
mainfrom
lpimentel/fix-stats-span-kind-peer-tags

Conversation

@lucaspimentel

@lucaspimentel lucaspimentel commented Apr 16, 2026

Copy link
Copy Markdown
Member

Overview

Configure SpanConcentrator in bottlecap with the Go agent's default ComputeStatsBySpanKind span kinds and basePeerTags peer tag keys, fixing two gaps in agent-side trace stats computation:

  • APMSVLS-464: span_kinds_stats_computed was empty, so non-top-level, non-measured spans with span.kind = server/client/producer/consumer) were silently excluded from stats.
  • APMSVLS-463: peer_tag_keys was empty, so client/producer/consumer spans had no per-dependency granularity in stats output (e.g., all S3 buckets, DynamoDB tables, Kafka topics lumped together).

Changes

  • Add STATS_ELIGIBLE_SPAN_KINDS constant (4 span kinds matching the Go agent's KindsComputed)
  • Add DEFAULT_PEER_TAG_KEYS constant (43 peer tag keys matching the Go agent's basePeerTags from mappings.json)
  • Pass both to SpanConcentrator::new() instead of empty vecs
  • Add unit tests that exercise the StatsConcentratorService pipeline for both features
  • Add fake-intake integration tests that route real spans through the concentrator → flusher → intake path, verifying span kind and peer tags survive msgpack/gzip serialization end to end

Follow-up

Both constants are hand-copied from the Go agent (KindsComputed and basePeerTags), which is the source of truth, and the same lists are also duplicated in serverless-components. A future refactor should keep the Rust agents in sync with the Go agent rather than each maintaining its own copy.

Testing

  • test_span_kind_stats_computed: sends a non-root, non-measured client span through the service, verifies stats are produced with span_kind="client"
  • test_peer_tags_populated: sends a client span with db.instance and db.system meta, verifies peer_tags contains both in the stats output
  • stats_span_kind_through_fake_intake: feeds a non-root, non-measured server span through the full concentrator → flusher → fake-intake path, asserts the decoded StatsPayload carries span_kind="server"
  • stats_peer_tags_through_fake_intake: feeds a client span with db.instance/db.system meta through the same path, asserts the decoded peer_tags contain both keys after serialization
  • Tests were written first and confirmed failing before the fix was applied
  • Full test suite still passes

"Turns out an empty vec is a great way to compute stats on absolutely nothing. Efficient, but not very useful." — Claude 🤖

@lucaspimentel lucaspimentel changed the title [APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats in bottlecap [APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats Apr 16, 2026
@lucaspimentel
lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from c06bc9b to 2eb4403 Compare June 10, 2026 17:58
@datadog-datadog-prod-us1

This comment has been minimized.

@lucaspimentel
lucaspimentel marked this pull request as ready for review June 10, 2026 21:49
@lucaspimentel
lucaspimentel requested review from a team as code owners June 10, 2026 21:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR configures Bottlecap’s SpanConcentrator to match the Go agent defaults for (1) which span.kind values are eligible for stats computation and (2) which peer-related tag keys are used to populate per-dependency peer_tags, closing gaps in agent-side trace stats aggregation.

Changes:

  • Configure SpanConcentrator with the Go agent’s default computed span kinds (client, server, producer, consumer) instead of an empty list.
  • Configure SpanConcentrator with the Go agent’s default peer tag key set (basePeerTags) instead of an empty list.
  • Add end-to-end tests for span-kind-based stats eligibility and peer tag population in emitted stats.

Comment thread bottlecap/src/traces/stats_concentrator_service.rs Outdated
Comment thread bottlecap/src/traces/stats_concentrator_service.rs Outdated
Comment thread bottlecap/src/traces/stats_concentrator_service.rs
Comment thread bottlecap/src/traces/stats_concentrator_service.rs Outdated
Comment thread bottlecap/src/traces/stats_concentrator_service.rs Outdated
@lucaspimentel
lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from 4ce8868 to 5e31b4f Compare June 11, 2026 22:34
@lucaspimentel lucaspimentel changed the title [APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats fix: Configure agent-side span kind and peer tag stats Jun 11, 2026
@lucaspimentel lucaspimentel changed the title fix: Configure agent-side span kind and peer tag stats fix: Configure span kind and peer tag for agent-side stats Jun 11, 2026
@lucaspimentel
lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from 5e31b4f to a652afd Compare June 12, 2026 21:48
@lucaspimentel lucaspimentel changed the title fix: Configure span kind and peer tag for agent-side stats fix(trace stats): Configure span kind and peer tag for agent-side stats Jun 12, 2026
@lucaspimentel
lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from 0584c5a to ebc24a6 Compare June 15, 2026 17:45
Tests verify that StatsConcentratorService computes stats
for spans with span.kind and populates peer_tags in output.
Both tests currently fail due to empty span_kinds_stats_computed
and peer_tag_keys vecs passed to SpanConcentrator::new().

🤖 Co-Authored-By: Claude Code <[email protected]>
Pass STATS_ELIGIBLE_SPAN_KINDS and DEFAULT_PEER_TAG_KEYS to
SpanConcentrator::new() to match the Go agent defaults. This
enables stats for OTel spans with span.kind and adds per-dependency
granularity via peer tags for client/producer/consumer spans.

🤖 Co-Authored-By: Claude Code <[email protected]>
🤖 Co-Authored-By: Claude Code <[email protected]>
Cover the concentrator's span-kind and peer-tag eligibility through the
full concentrator -> flusher -> intake path, verifying both survive
msgpack/gzip serialization.
@lucaspimentel
lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from ebc24a6 to aad95a3 Compare June 15, 2026 21:03
@lucaspimentel
lucaspimentel merged commit 4caba79 into main Jun 15, 2026
15 of 16 checks passed
@lucaspimentel
lucaspimentel deleted the lpimentel/fix-stats-span-kind-peer-tags branch June 15, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants