Skip to content

fix(profiling): crash in ddtrace_get_profiling_context#3563

Merged
morrisonlevi merged 1 commit intomasterfrom
levi/ddtrace_get_profiling_context
Jan 12, 2026
Merged

fix(profiling): crash in ddtrace_get_profiling_context#3563
morrisonlevi merged 1 commit intomasterfrom
levi/ddtrace_get_profiling_context

Conversation

@morrisonlevi
Copy link
Copy Markdown
Collaborator

@morrisonlevi morrisonlevi commented Jan 11, 2026

Description

This crash shows that if a span is closed and an allocation takes place, then it's possible for the allocation profiler to dereference a null pointer. GENERALLY, this pointer is not null, it's just a sort of weird edge:

#0   0x00007f9765f9a37c ddtrace_get_profiling_context (ext/profiling.c:12)
#1   0x00007f97654a5a68 common_labels (src/profiling/mod.rs:1446:32)
#2   0x00007f9765466e1e collect_allocations (src/profiling/mod.rs:953:30)
#3   0x00007f9765466e1e collect_allocation (allocation/mod.rs:77:13)
#4   0x00007f97654a827f alloc_prof_malloc (allocation/allocation_le83.rs:361:9)
#5   0x0000564703741654 smart_str_erealloc 
#6   0x000056470354c5fb php_json_encode_ex 
#7   0x00007f9765f9969e ddtrace_decide_on_closed_span_sampling (priority_sampling/priority_sampling.c:187)
#8   0x00007f9765fa78d9 ddtrace_close_top_span_without_stack_swap (ext/span.c:965)
#9   0x00007f9765fa7ebb ddtrace_close_span (ext/span.c:912)
#10  0x00007f9765fa849b ddtrace_clear_execute_data_span (ext/span.c:562)
#11  0x00007f9765fb137a dd_uhook_end (hook/uhook_legacy.c:246)
#12  0x00007f9765f55515 zai_hook_finish (hook/hook.c:1116)
#13  0x00007f9765f46e5e zai_hook_safe_finish (php8/interceptor.c:56)
#14  0x00007f9765f46f83 zai_interceptor_observer_end_handler (php8/interceptor.c:176)
#15  0x0000564703742be4 zend_observer_fcall_end 
#16  0x000056470370c89c execute_ex 
#17  0x0000564703716885 zend_execute 
#18  0x00005647036a0f10 zend_execute_scripts 
#19  0x00005647036347fa php_execute_script 
#20  0x00007f9772514e40 __libc_start_main 
#21  0x00005647034c2b65 _start 

You can use this code with a datadog.profiling.allocation_sample_distance=1 to verify this, but I struggled to make a test and reproduce this without patching code:

diff --git a/ext/span.c b/ext/span.c
index 000000000..111111111 100644
--- a/ext/span.c
+++ b/ext/span.c
@@ -825,10 +825,25 @@ static void dd_close_entry_span_of_stack(ddtrace_span_stack *stack) {
     if (!stack->root_span || stack->root_span->stack == stack) {
         // Ensure the root span is cleared before allocations may happen in priority sampling deciding
         ddtrace_root_span_data *root_span = stack->root_span;
         if (stack->root_span) {
+            // Reproducer assist: if allocation profiling calls back into ddtrace while the trace root
+            // is closing, `stack->active` can already be NULL, while `stack->root_span` is still set.
+            // Force a small burst of allocations in that window so the crash (NULL deref in
+            // ddtrace_get_profiling_context on affected versions) becomes much easier to hit.
+            //
+            // Only enabled when ddtrace debug is on, to avoid impacting normal runs.
+            if (UNEXPECTED(get_DD_TRACE_DEBUG())) {
+                for (int i = 0; i < 10000; i++) {
+                    zend_string *tmp = zend_string_init(ZEND_STRL("profctx-repro"), 0);
+                    zend_string_release(tmp);
+                }
+            }
+
             // Root span stacks are automatic and tied to the lifetime of that root
             stack->root_span = NULL;
 
             // Enforce a sampling decision here
             ddtrace_fetch_priority_sampling_from_span(root_span);
         }

This crash was detected roughly 786 in the past week.

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@datadog-datadog-prod-us1
Copy link
Copy Markdown

datadog-datadog-prod-us1 Bot commented Jan 11, 2026

⚠️ Tests

Fix all issues with Cursor

⚠️ Warnings

❄️ 4 New flaky tests detected

testLoggedInCalls from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest::testLoggedInCalls
Failed asserting that 0 matches expected 1.

tests/Integrations/Laravel/AutomatedLoginEventsTestSuite.php:112
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testUserLoginFailureEvent from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest::testUserLoginFailureEvent
Failed asserting that 0 matches expected 1.

tests/Integrations/Laravel/AutomatedLoginEventsTestSuite.php:62
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testUserLoginSuccessEvent from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest::testUserLoginSuccessEvent
Failed asserting that 0 matches expected 1.

tests/Integrations/Laravel/AutomatedLoginEventsTestSuite.php:48
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testUserSignUp from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\AutomatedLoginEventsTest::testUserSignUp
Failed asserting that 0 matches expected 1.

tests/Integrations/Laravel/AutomatedLoginEventsTestSuite.php:78
tests/Common/RetryTraitVersionGeneric.php:28
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
View all

🧪 2141 Tests failed

    testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Fix with Cursor)

    testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Fix with Cursor)

testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 6964220400000000d504fa0c04e4807f
tid: 6964220400000000
hexProcessTraceId: d504fa0c04e4807f
hexProcessSpanId: a8c5787361f153df
processTraceId: 15349668359514128511
processSpanId: 12161258805743670239

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
View all
This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1f37473 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@morrisonlevi morrisonlevi marked this pull request as ready for review January 11, 2026 22:29
@morrisonlevi morrisonlevi requested a review from a team as a code owner January 11, 2026 22:29
@morrisonlevi morrisonlevi merged commit 81e8ab5 into master Jan 12, 2026
2005 of 2007 checks passed
@morrisonlevi morrisonlevi deleted the levi/ddtrace_get_profiling_context branch January 12, 2026 16:41
@github-actions github-actions Bot added this to the 1.16.0 milestone Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants