Skip to content

SLES-2898 Fix cold start timeout spans#1254

Merged
duncanista merged 1 commit into
mainfrom
dd/sles-2898-cold-start-timeout-spans
Jun 12, 2026
Merged

SLES-2898 Fix cold start timeout spans#1254
duncanista merged 1 commit into
mainfrom
dd/sles-2898-cold-start-timeout-spans

Conversation

@duncanista

Copy link
Copy Markdown
Contributor

Overview

Motivation: Customers without an application tracer can hit Lambda timeouts where the extension synthesizes an aws.lambda trace ID for timeout handling. That generated trace ID was also copied onto the cold start span, making the extension send a cold start span even though no tracer had established an APM trace.

Changes:

  • Only copy the invocation trace ID to the cold start span when tracer instrumentation was detected.
  • Preserve the Node/Python behavior where aws.lambda.load sets the cold start trace ID directly before the cold start span is sent.
  • Added regression coverage for timeout handling without a tracer and for preserving tracer-provided cold start trace IDs.

Testing

  • RUSTUP_TOOLCHAIN=stable cargo fmt --check
  • Attempted RUSTUP_TOOLCHAIN=stable cargo test --lib cold_start --no-default-features; blocked before compilation because the sandbox allowlist returned 403 fetching DataDog/serverless-components for the datadog-fips git dependency.

PR by Bits - View session in Datadog

Comment @DataDog to request changes

@datadog-official

datadog-official Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

View session in Datadog

Bits Code status: ✅ Done

Comment @DataDog to request changes

Copilot AI review requested due to automatic review settings June 12, 2026 17:40
@duncanista
duncanista requested a review from a team as a code owner June 12, 2026 17:40
@duncanista
duncanista requested a review from lym953 June 12, 2026 17:40
@datadog-datadog-prod-us1

Copy link
Copy Markdown

I can only run on private repositories.

@duncanista
duncanista merged commit 7afe441 into main Jun 12, 2026
16 of 18 checks passed
@duncanista
duncanista deleted the dd/sles-2898-cold-start-timeout-spans branch June 12, 2026 17:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an edge case where timeout handling can synthesize an invocation trace ID and inadvertently make the cold start span “sendable” even when no application tracer established an APM trace (notably impacting non-universally-instrumented runtimes). The change ensures cold start spans are only tied to an invocation trace when tracer instrumentation is actually detected, while preserving the Node/Python aws.lambda.load behavior of explicitly setting the cold start trace ID.

Changes:

  • Gate copying the invocation trace/parent IDs onto the cold start span behind tracer_detected to avoid timeout-synthesized IDs making cold start spans eligible to send.
  • Add regression tests covering timeout handling without a tracer and preserving tracer-provided cold start trace IDs.

lucaspimentel added a commit that referenced this pull request Jun 15, 2026
## Overview

Two tests added in #1254 were failing on `main`. The Rust "Test Suite"
CI job is not a required check (only `devflow/mergegate` is), so the red
suite did not block #1254 from merging.

### 1.
`enrich_ctx_keeps_tracer_set_cold_start_trace_id_without_tracer_detected`
Annotated `#[test]` but calls `setup()`, which spawns a Tokio task via
`tokio::spawn(service.run())`. Running it panicked with `there is no
reactor running, must be called from the context of a Tokio 1.x
runtime`. Changed to `#[tokio::test] async fn` to match its sibling
tests.

### 2. `timeout_without_tracer_does_not_send_cold_start_span`
Passed its `trace_sender` by value into `on_platform_runtime_done`,
dropping the only `Arc<SendingTraceProcessor>` before `try_recv()`. The
receiver then reported `Disconnected`, but the test asserted `Empty`, so
it failed even though the production code correctly never sent the cold
start span. Fixed by passing a clone and holding the sender until after
the assertion so the channel stays connected.

No production code changes; both fixes are test-only.

> *"A test that drops its own sender and then complains the line went
dead: the unit-testing equivalent of cutting the phone cord and filing a
complaint about the silence."* — Claude 🤖
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants