SLES-2898 Fix cold start timeout spans#1254
Merged
Merged
Conversation
Contributor
|
Bits Code status: ✅ Done Comment @DataDog to request changes |
|
I can only run on private repositories. |
litianningdatadog
approved these changes
Jun 12, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes an edge case where timeout handling can synthesize an invocation trace ID and inadvertently make the cold start span “sendable” even when no application tracer established an APM trace (notably impacting non-universally-instrumented runtimes). The change ensures cold start spans are only tied to an invocation trace when tracer instrumentation is actually detected, while preserving the Node/Python aws.lambda.load behavior of explicitly setting the cold start trace ID.
Changes:
- Gate copying the invocation trace/parent IDs onto the cold start span behind
tracer_detectedto avoid timeout-synthesized IDs making cold start spans eligible to send. - Add regression tests covering timeout handling without a tracer and preserving tracer-provided cold start trace IDs.
lucaspimentel
added a commit
that referenced
this pull request
Jun 15, 2026
## Overview Two tests added in #1254 were failing on `main`. The Rust "Test Suite" CI job is not a required check (only `devflow/mergegate` is), so the red suite did not block #1254 from merging. ### 1. `enrich_ctx_keeps_tracer_set_cold_start_trace_id_without_tracer_detected` Annotated `#[test]` but calls `setup()`, which spawns a Tokio task via `tokio::spawn(service.run())`. Running it panicked with `there is no reactor running, must be called from the context of a Tokio 1.x runtime`. Changed to `#[tokio::test] async fn` to match its sibling tests. ### 2. `timeout_without_tracer_does_not_send_cold_start_span` Passed its `trace_sender` by value into `on_platform_runtime_done`, dropping the only `Arc<SendingTraceProcessor>` before `try_recv()`. The receiver then reported `Disconnected`, but the test asserted `Empty`, so it failed even though the production code correctly never sent the cold start span. Fixed by passing a clone and holding the sender until after the assertion so the channel stays connected. No production code changes; both fixes are test-only. > *"A test that drops its own sender and then complains the line went dead: the unit-testing equivalent of cutting the phone cord and filing a complaint about the silence."* — Claude 🤖
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Motivation: Customers without an application tracer can hit Lambda timeouts where the extension synthesizes an
aws.lambdatrace ID for timeout handling. That generated trace ID was also copied onto the cold start span, making the extension send a cold start span even though no tracer had established an APM trace.Changes:
aws.lambda.loadsets the cold start trace ID directly before the cold start span is sent.Testing
RUSTUP_TOOLCHAIN=stable cargo fmt --checkRUSTUP_TOOLCHAIN=stable cargo test --lib cold_start --no-default-features; blocked before compilation because the sandbox allowlist returned 403 fetchingDataDog/serverless-componentsfor thedatadog-fipsgit dependency.PR by Bits - View session in Datadog
Comment @DataDog to request changes