It is a implementation detail in Ruby that Thread.new will raise ThreadError: can't alloc thread if the process has already called exit. If the user has threads executing long-running jobs, those threads will be killed by an Interrupt. But if the user has an ensure block which is triggered by that error, that block can trigger a metric flush causing DataDog's sender Thread to fail to start.
The code to create the Thread is here:
|
@sender_thread = @thread_class.new(&method(:send_loop)) |
I would suggest that DataDog rescue and ignore this ThreadError. It is a rare edge case and any last minute flushes could be treated as "best effort" and not guaranteed. Alternatively the current thread could be used to flush if a ThreadError occurs. What do you think?
Original issue here: sidekiq/sidekiq#6603 (comment)
It is a implementation detail in Ruby that
Thread.newwill raiseThreadError: can't alloc threadif the process has already calledexit. If the user has threads executing long-running jobs, those threads will be killed by an Interrupt. But if the user has an ensure block which is triggered by that error, that block can trigger a metric flush causing DataDog's sender Thread to fail to start.The code to create the Thread is here:
dogstatsd-ruby/lib/datadog/statsd/sender.rb
Line 100 in e548be0
I would suggest that DataDog rescue and ignore this ThreadError. It is a rare edge case and any last minute flushes could be treated as "best effort" and not guaranteed. Alternatively the current thread could be used to flush if a ThreadError occurs. What do you think?
Original issue here: sidekiq/sidekiq#6603 (comment)