fix(curl): enforce DD_TRACE_SPANS_LIMIT for curl_multi_exec parent spans (APMS-19944)#4030
Conversation
…ans (APMS-19944) The curl_multi_exec integration opens its parent span via the user-facing DDTrace\start_span(), which intentionally bypasses DD_TRACE_SPANS_LIMIT (see #3691). Distributed-header injection then flags that span NOT_DROPPABLE, so the post-hook's try_drop_span() can never reclaim it. In long-running CLI workloads that repeatedly call curl_multi_exec (e.g. AWS SDK SQS batches), one un-droppable span accumulates per call, growing memory 1:1 with calls until OOM. Skip creating the parent span when the tracer is already limited. Distributed tracing headers are still injected by the C handler against the active span, so propagation is preserved, and the child request spans are already limited. Fixes: APMS-19944
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 06bcdf2 | Docs | Datadog PR Page | Give us feedback! |
dd_trace_tracer_is_limited() reflects the open+closed span counters at the moment of the call, which depends on end-of-run flush/cleanup timing and varies across CI environments and PHP versions (7.3/7.4/8.2-ASAN all reported bool(false) at script end). The span-count assertion (BOUNDED) is the real regression guard: with the fix the retained curl_multi_exec spans stay capped near DD_TRACE_SPANS_LIMIT, without it they scale 1:1 with iterations. Bump to 100 iterations for a clearer margin and drop the fragile check.
Benchmarks [ tracer ]Benchmark execution time: 2026-07-03 16:04:36 Comparing candidate commit 06bcdf2 in PR branch Found 0 performance improvements and 5 performance regressions! Performance is the same for 189 metrics, 0 unstable metrics.
|
Address review: move the ddtrace_tracer_is_limited() guard to the start of the begin hook so the limited path returns immediately, skipping the args lookup and ObjectKVStore reuse check. Spans for a multi-op are only created on the first curl_multi_exec call, so a new handle has nothing to reuse when limited; the boundary case (parent created just before the limit) is still finalized by CurlSpanInfo::__destruct().
2 participants
What
Skip creating the
curl_multi_execparent span when the tracer is already limited (DD_TRACE_SPANS_LIMITreached / memory pressure).Why (APMS-19944)
Follow-up to #3691 (APMS-18744). That PR limited the child header-injection spans but left the parent
curl_multi_execspan unbounded.The parent span is opened via the user-facing
DDTrace\start_span()(CurlIntegration.php), which #3691 deliberately left exempt from the span limit (guarding it broketestTracerFlushedWhenSpanLimitExceededand Guzzle'sisolateTracer()). Distributed-header injection then flags that spanDDTRACE_SPAN_FLAG_NOT_DROPPABLE(handlers_http.h), so the post-hook'stry_drop_span()can never reclaim it.In long-running CLI workloads that repeatedly call
curl_multi_exec(e.g. AWS SDK SQSsendMessagebatches), one un-droppable parent span accumulates per call, growing memory 1:1 with calls until OOM.Reported: PHP 8.2, ECS Fargate ARM64, reproduced on 1.17.0 and 1.22.0 — 10,000 iterations → ~10,500 spans in one trace.
Fix
Gate the internal integration use of
start_span()withdd_trace_tracer_is_limited(), without touching the user-facing API (so the tests behind the #3691 revert stay green). When limited, the C handler still injects distributed tracing headers against the active span, so propagation is preserved; child request spans are already limited.Added regression test
tests/ext/integrations/curl/curl_multi_exec_span_limit.phpt— PASS with the fix, FAIL (LEAK) without it.Fixes: APMS-19944