Good morning!
I found this bug in crash logs for one of our service and I believe it is caused by ddprof.
Error UnixSignal: Process terminated with SEGV_MAPERR (SIGSEGV)
#0 0x0000f44c544bff98 ddprof::SpinLock::try_lock_until_slow
#1 0x0000f44c544bf4b8 ddprof::AllocationTracker::push_alloc_sample
#2 0x0000f44c544bf8c4 ddprof::AllocationTracker::track_allocation
#3 0x0000f44c544c4b2c ddprof::(anonymous namespace)::MmapHook::hook
#4 0x0000f44ba813f598 libdd_alloc::chain::ChainAllocator<A>::grow::h8cb1e915e90024d1
#5 0x0000f44ba8323778 <libdd_alloc::chain::ChainAllocator<A> as allocator_api2::stable::alloc::Allocator>::allocate::h095b386cb13219aa
#6 0x0000f44ba833ed2c libdd_profiling::profiles::collections::slice_set::SliceSet<T>::insert_unique_uncontended_with_hash::h569d502dc6443e73
#7 0x0000f44ba8246900 ddog_prof_ProfilesDictionary_insert_str
#8 0x0000f44b9662b44c Datadog::intern_string
#9 0x0000f44b964c0bb0 Datadog::StackRenderer::render_frame
#10 0x0000f44b964c0e40 FrameStack::render
#11 0x0000f44b964c1a78 ThreadInfo::sample
#12 0x0000f44b964c1aac std::_Function_handler<void (_ts*, ThreadInfo&), Datadog::Sampler::sampling_thread(unsigned long)::{lambda(InterpreterInfo&)#1}::operator()(InterpreterInfo&) const::{lambda(_ts*, ThreadInfo&)#1}>::_M_invoke
#13 0x0000f44b964becd8 for_each_thread
#14 0x0000f44b964bed4c std::_Function_handler<void (InterpreterInfo&), Datadog::Sampler::sampling_thread(unsigned long)::{lambda(InterpreterInfo&)#1}>::_M_invoke
#15 0x0000f44b964bc180 for_each_interp
#16 0x0000f44b964bf0c8 Datadog::Sampler::sampling_thread
#17 0x0000f44b964bf290 call_sampling_thread
It's a bit misleading because frames from the dd-trace-py Profiler and ddprof itself, I think the relevant part really just is
#0 0x0000f44c544bff98 ddprof::SpinLock::try_lock_until_slow
#1 0x0000f44c544bf4b8 ddprof::AllocationTracker::push_alloc_sample
#2 0x0000f44c544bf8c4 ddprof::AllocationTracker::track_allocation
#3 0x0000f44c544c4b2c ddprof::(anonymous namespace)::MmapHook::hook
#4 0x0000f44ba813f598 libdd_alloc::chain::ChainAllocator<A>::grow::h8cb1e915e90024d1
#5 0x0000f44ba8323778 <libdd_alloc::chain::ChainAllocator<A> as allocator_api2::stable::alloc::Allocator>::allocate::h095b386cb13219aa
I don't know much about the ddprof codebase but after debugging with an LLM, I believe I have a credible root cause, I'll post it here after double checking/validating more.
Good morning!
I found this bug in crash logs for one of our service and I believe it is caused by
ddprof.It's a bit misleading because frames from the
dd-trace-pyProfiler andddprofitself, I think the relevant part really just isI don't know much about the
ddprofcodebase but after debugging with an LLM, I believe I have a credible root cause, I'll post it here after double checking/validating more.