perf(internal, profiling): native PeriodicThread avoids awake() after fork() (#16545) [backport 4.5]#16900
Conversation
…er fork() After #14163, we started to see a profiler behavior change in `dd-trace-doe` CI: - `TestLanguage/python/flush/false` expected `0` profiles, got `3`. https://github.com/DataDog/dd-trace-doe/actions/runs/22102391228/job/63875669025 - During pre-fork, `PeriodicThread.stop()` sets the internal request event to wake the periodic threads promptly. - After fork restart, that same request could be interpreted as a real wakeup, causing an immediate `periodic()` run. - For profiling, that means an immediate `upload()` right after fork, which results in unexpected profile exports. What this PR changfes: - Distinguishes a fork-stop wakeup from a real `awake()` request in `PeriodicThread` - Clears the request after fork only when it came from the pre-fork stop path. - Preserves real `awake()` behavior across restart windows. <!-- Describe your testing strategy or note what tests are included --> <!-- Note any risks associated with this change, or "None" if no risks --> <!-- Any other information that would be helpful for reviewers --> Co-authored-by: taegyun.kim <[email protected]> (cherry picked from commit 295a47f)
Codeowners resolved as |
PeriodicThread avoids awake() after fork() (#16545)PeriodicThread avoids awake() after fork() (#16545) [backport 4.5]
Performance SLOsComparing candidate kowalski/backport-16545-to-4.5 (6cac384) with baseline 4.5 (c3478bd) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 102.931µs (SLO: <130.000µs 📉 -20.8%) vs baseline: +2.8% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ add_inplace_aspectTime: ✅ 100.413µs (SLO: <130.000µs 📉 -22.8%) vs baseline: -0.8% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ add_inplace_noaspectTime: ✅ 28.273µs (SLO: <40.000µs 📉 -29.3%) vs baseline: ~same Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ add_noaspectTime: ✅ 48.810µs (SLO: <70.000µs 📉 -30.3%) vs baseline: +0.4% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 250.729µs (SLO: <400.000µs 📉 -37.3%) vs baseline: +0.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.9% ✅ bytearray_extend_aspectTime: ✅ 639.303µs (SLO: <800.000µs 📉 -20.1%) vs baseline: +0.4% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ bytearray_extend_noaspectTime: ✅ 263.766µs (SLO: <400.000µs 📉 -34.1%) vs baseline: +0.9% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ bytearray_noaspectTime: ✅ 134.382µs (SLO: <300.000µs 📉 -55.2%) vs baseline: -0.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ bytes_aspectTime: ✅ 218.030µs (SLO: <300.000µs 📉 -27.3%) vs baseline: -0.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ bytes_noaspectTime: ✅ 133.882µs (SLO: <200.000µs 📉 -33.1%) vs baseline: +0.5% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ bytesio_aspectTime: ✅ 3.792ms (SLO: <5.000ms 📉 -24.2%) vs baseline: +0.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ bytesio_noaspectTime: ✅ 317.508µs (SLO: <420.000µs 📉 -24.4%) vs baseline: +0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ capitalize_aspectTime: ✅ 89.326µs (SLO: <300.000µs 📉 -70.2%) vs baseline: ~same Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ capitalize_noaspectTime: ✅ 250.970µs (SLO: <300.000µs 📉 -16.3%) vs baseline: +0.5% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +5.0% ✅ casefold_aspectTime: ✅ 89.424µs (SLO: <500.000µs 📉 -82.1%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% ✅ casefold_noaspectTime: ✅ 304.511µs (SLO: <500.000µs 📉 -39.1%) vs baseline: -1.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ decode_aspectTime: ✅ 86.371µs (SLO: <100.000µs 📉 -13.6%) vs baseline: -0.3% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ decode_noaspectTime: ✅ 153.420µs (SLO: <210.000µs 📉 -26.9%) vs baseline: +0.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.7% ✅ encode_aspectTime: ✅ 84.553µs (SLO: <200.000µs 📉 -57.7%) vs baseline: +0.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ encode_noaspectTime: ✅ 140.569µs (SLO: <200.000µs 📉 -29.7%) vs baseline: +1.1% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +5.0% ✅ format_aspectTime: ✅ 14.614ms (SLO: <19.200ms 📉 -23.9%) vs baseline: +0.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.8% ✅ format_map_aspectTime: ✅ 16.386ms (SLO: <21.500ms 📉 -23.8%) vs baseline: -0.2% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.9% ✅ format_map_noaspectTime: ✅ 371.087µs (SLO: <500.000µs 📉 -25.8%) vs baseline: +0.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +5.0% ✅ format_noaspectTime: ✅ 302.767µs (SLO: <500.000µs 📉 -39.4%) vs baseline: +0.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +5.0% ✅ index_aspectTime: ✅ 128.441µs (SLO: <300.000µs 📉 -57.2%) vs baseline: +5.5% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ index_noaspectTime: ✅ 40.429µs (SLO: <300.000µs 📉 -86.5%) vs baseline: +0.3% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ join_aspectTime: ✅ 210.147µs (SLO: <300.000µs 📉 -30.0%) vs baseline: +0.2% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +5.0% ✅ join_noaspectTime: ✅ 143.677µs (SLO: <300.000µs 📉 -52.1%) vs baseline: +1.2% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ ljust_aspectTime: ✅ 581.818µs (SLO: <700.000µs 📉 -16.9%) vs baseline: 📈 +16.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ ljust_noaspectTime: ✅ 258.014µs (SLO: <300.000µs 📉 -14.0%) vs baseline: -0.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.7% ✅ lower_aspectTime: ✅ 298.470µs (SLO: <500.000µs 📉 -40.3%) vs baseline: +0.9% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ lower_noaspectTime: ✅ 234.071µs (SLO: <300.000µs 📉 -22.0%) vs baseline: -1.1% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ lstrip_aspectTime: ✅ 0.271ms (SLO: <3.000ms 📉 -91.0%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.9% ✅ lstrip_noaspectTime: ✅ 0.179ms (SLO: <3.000ms 📉 -94.0%) vs baseline: +0.8% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 14.394ms (SLO: <18.750ms 📉 -23.2%) vs baseline: +0.7% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.8% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.790ms (SLO: <19.350ms 📉 -23.6%) vs baseline: -1.2% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 14.537ms (SLO: <18.900ms 📉 -23.1%) vs baseline: +1.3% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.566ms (SLO: <19.150ms 📉 -23.9%) vs baseline: -0.3% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ modulo_noaspectTime: ✅ 0.362ms (SLO: <3.000ms 📉 -87.9%) vs baseline: +1.0% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ replace_aspectTime: ✅ 18.396ms (SLO: <24.000ms 📉 -23.4%) vs baseline: -0.2% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.7% ✅ replace_noaspectTime: ✅ 281.287µs (SLO: <300.000µs -6.2%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ repr_aspectTime: ✅ 314.462µs (SLO: <420.000µs 📉 -25.1%) vs baseline: +0.4% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.9% ✅ repr_noaspectTime: ✅ 46.689µs (SLO: <90.000µs 📉 -48.1%) vs baseline: -0.6% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ rstrip_aspectTime: ✅ 385.910µs (SLO: <500.000µs 📉 -22.8%) vs baseline: +2.0% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ rstrip_noaspectTime: ✅ 184.956µs (SLO: <300.000µs 📉 -38.3%) vs baseline: +0.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.1% ✅ slice_aspectTime: ✅ 186.590µs (SLO: <300.000µs 📉 -37.8%) vs baseline: +0.4% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +4.7% ✅ slice_noaspectTime: ✅ 54.303µs (SLO: <90.000µs 📉 -39.7%) vs baseline: +0.5% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.8% ✅ stringio_aspectTime: ✅ 4.400ms (SLO: <5.000ms 📉 -12.0%) vs baseline: 📈 +15.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ stringio_noaspectTime: ✅ 346.585µs (SLO: <500.000µs 📉 -30.7%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ strip_aspectTime: ✅ 269.946µs (SLO: <350.000µs 📉 -22.9%) vs baseline: +0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ strip_noaspectTime: ✅ 176.340µs (SLO: <240.000µs 📉 -26.5%) vs baseline: -0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ swapcase_aspectTime: ✅ 331.465µs (SLO: <500.000µs 📉 -33.7%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ swapcase_noaspectTime: ✅ 272.566µs (SLO: <400.000µs 📉 -31.9%) vs baseline: +0.7% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ title_aspectTime: ✅ 321.594µs (SLO: <500.000µs 📉 -35.7%) vs baseline: -0.4% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ title_noaspectTime: ✅ 257.482µs (SLO: <400.000µs 📉 -35.6%) vs baseline: -0.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ translate_aspectTime: ✅ 492.722µs (SLO: <700.000µs 📉 -29.6%) vs baseline: -0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 421.117µs (SLO: <500.000µs 📉 -15.8%) vs baseline: -0.9% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +5.1% ✅ upper_aspectTime: ✅ 299.273µs (SLO: <500.000µs 📉 -40.1%) vs baseline: +0.8% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.8% ✅ upper_noaspectTime: ✅ 236.467µs (SLO: <400.000µs 📉 -40.9%) vs baseline: +0.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 512.401µs (SLO: <700.000µs 📉 -26.8%) vs baseline: 📈 +21.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.6% ✅ ospathbasename_noaspectTime: ✅ 433.544µs (SLO: <700.000µs 📉 -38.1%) vs baseline: -0.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +5.6% ✅ ospathjoin_aspectTime: ✅ 627.724µs (SLO: <700.000µs 📉 -10.3%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.4% ✅ ospathjoin_noaspectTime: ✅ 634.309µs (SLO: <700.000µs -9.4%) vs baseline: ~same Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 350.094µs (SLO: <700.000µs 📉 -50.0%) vs baseline: -0.8% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +5.4% ✅ ospathnormcase_noaspectTime: ✅ 358.568µs (SLO: <700.000µs 📉 -48.8%) vs baseline: +0.4% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +5.2% ✅ ospathsplit_aspectTime: ✅ 489.080µs (SLO: <700.000µs 📉 -30.1%) vs baseline: ~same Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% ✅ ospathsplit_noaspectTime: ✅ 501.432µs (SLO: <700.000µs 📉 -28.4%) vs baseline: +0.3% Memory: ✅ 42.821MB (SLO: <46.000MB -6.9%) vs baseline: +4.7% ✅ ospathsplitdrive_aspectTime: ✅ 376.950µs (SLO: <700.000µs 📉 -46.1%) vs baseline: +0.1% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.5% ✅ ospathsplitdrive_noaspectTime: ✅ 73.254µs (SLO: <700.000µs 📉 -89.5%) vs baseline: +0.2% Memory: ✅ 42.723MB (SLO: <46.000MB -7.1%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 459.761µs (SLO: <700.000µs 📉 -34.3%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.6% ✅ ospathsplitext_noaspectTime: ✅ 466.510µs (SLO: <700.000µs 📉 -33.4%) vs baseline: +0.4% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +5.3% 🟡 Near SLO Breach (1 suite)🟡 tracer - 6/6✅ largeTime: ✅ 31.606ms (SLO: <32.950ms -4.1%) vs baseline: +0.2% Memory: ✅ 36.746MB (SLO: <39.250MB -6.4%) vs baseline: +5.0% ✅ mediumTime: ✅ 3.107ms (SLO: <3.200ms -2.9%) vs baseline: -0.1% Memory: ✅ 35.527MB (SLO: <38.750MB -8.3%) vs baseline: +5.1% ✅ smallTime: ✅ 363.906µs (SLO: <370.000µs 🟡 -1.6%) vs baseline: +4.4% Memory: ✅ 35.488MB (SLO: <38.750MB -8.4%) vs baseline: +4.6%
|
|
/merge -f --reason "need to release a patch" |
|
View all feedbacks in Devflow UI.
Arguments errors:
If you need support, contact us on Slack #devflow with those details! |
…6901) ## Description This is a backport for #16741 to 4.5. Depends on #16900. --------- Co-authored-by: Taegyun Kim <[email protected]> Co-authored-by: Gabriele N. Tornetta <[email protected]>
Description
Backport of #16545 to 4.5.