fix(celery): distributed tracing for consuming unregistered tasks#16662
Conversation
This comment has been minimized.
This comment has been minimized.
Performance SLOsComparing candidate munir/always-inject-headers-celry-tasks (d7c2656) with baseline main (a6dfd8e) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 103.834µs (SLO: <130.000µs 📉 -20.1%) vs baseline: +1.0% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ add_inplace_aspectTime: ✅ 100.540µs (SLO: <130.000µs 📉 -22.7%) vs baseline: -2.8% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ add_inplace_noaspectTime: ✅ 28.270µs (SLO: <40.000µs 📉 -29.3%) vs baseline: +0.7% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ add_noaspectTime: ✅ 48.995µs (SLO: <70.000µs 📉 -30.0%) vs baseline: -0.3% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ bytearray_aspectTime: ✅ 251.678µs (SLO: <400.000µs 📉 -37.1%) vs baseline: +1.6% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ bytearray_extend_aspectTime: ✅ 637.220µs (SLO: <800.000µs 📉 -20.3%) vs baseline: -2.7% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ bytearray_extend_noaspectTime: ✅ 263.187µs (SLO: <400.000µs 📉 -34.2%) vs baseline: -4.9% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ bytearray_noaspectTime: ✅ 136.938µs (SLO: <300.000µs 📉 -54.4%) vs baseline: -3.8% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ bytes_aspectTime: ✅ 217.743µs (SLO: <300.000µs 📉 -27.4%) vs baseline: -0.9% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +4.1% ✅ bytes_noaspectTime: ✅ 133.341µs (SLO: <200.000µs 📉 -33.3%) vs baseline: -1.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ bytesio_aspectTime: ✅ 3.774ms (SLO: <5.000ms 📉 -24.5%) vs baseline: -0.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ bytesio_noaspectTime: ✅ 316.966µs (SLO: <420.000µs 📉 -24.5%) vs baseline: -1.1% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ capitalize_aspectTime: ✅ 89.318µs (SLO: <300.000µs 📉 -70.2%) vs baseline: +1.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ capitalize_noaspectTime: ✅ 249.496µs (SLO: <300.000µs 📉 -16.8%) vs baseline: -0.8% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.0% ✅ casefold_aspectTime: ✅ 93.141µs (SLO: <500.000µs 📉 -81.4%) vs baseline: +5.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ casefold_noaspectTime: ✅ 306.754µs (SLO: <500.000µs 📉 -38.6%) vs baseline: -1.8% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ decode_aspectTime: ✅ 86.605µs (SLO: <100.000µs 📉 -13.4%) vs baseline: -0.6% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.5% ✅ decode_noaspectTime: ✅ 151.400µs (SLO: <210.000µs 📉 -27.9%) vs baseline: -2.1% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ encode_aspectTime: ✅ 83.880µs (SLO: <200.000µs 📉 -58.1%) vs baseline: -1.2% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ encode_noaspectTime: ✅ 140.079µs (SLO: <200.000µs 📉 -30.0%) vs baseline: -2.5% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% ✅ format_aspectTime: ✅ 14.672ms (SLO: <19.200ms 📉 -23.6%) vs baseline: +0.2% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.2% ✅ format_map_aspectTime: ✅ 16.419ms (SLO: <21.500ms 📉 -23.6%) vs baseline: +0.4% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +4.8% ✅ format_map_noaspectTime: ✅ 369.716µs (SLO: <500.000µs 📉 -26.1%) vs baseline: -1.9% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.2% ✅ format_noaspectTime: ✅ 304.824µs (SLO: <500.000µs 📉 -39.0%) vs baseline: +0.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.1% ✅ index_aspectTime: ✅ 121.708µs (SLO: <300.000µs 📉 -59.4%) vs baseline: +1.9% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ index_noaspectTime: ✅ 40.322µs (SLO: <300.000µs 📉 -86.6%) vs baseline: -1.0% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.0% ✅ join_aspectTime: ✅ 210.203µs (SLO: <300.000µs 📉 -29.9%) vs baseline: -2.0% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ join_noaspectTime: ✅ 142.173µs (SLO: <300.000µs 📉 -52.6%) vs baseline: -1.2% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ ljust_aspectTime: ✅ 502.212µs (SLO: <700.000µs 📉 -28.3%) vs baseline: -1.2% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ ljust_noaspectTime: ✅ 256.667µs (SLO: <300.000µs 📉 -14.4%) vs baseline: -1.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ lower_aspectTime: ✅ 299.291µs (SLO: <500.000µs 📉 -40.1%) vs baseline: -1.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ lower_noaspectTime: ✅ 239.824µs (SLO: <300.000µs 📉 -20.1%) vs baseline: +1.6% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ lstrip_aspectTime: ✅ 0.268ms (SLO: <3.000ms 📉 -91.1%) vs baseline: -4.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ lstrip_noaspectTime: ✅ 0.212ms (SLO: <3.000ms 📉 -92.9%) vs baseline: 📈 +19.2% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ modulo_aspectTime: ✅ 14.433ms (SLO: <18.750ms 📉 -23.0%) vs baseline: +0.3% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.3% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.796ms (SLO: <19.350ms 📉 -23.5%) vs baseline: +0.3% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.3% ✅ modulo_aspect_for_bytesTime: ✅ 14.307ms (SLO: <18.900ms 📉 -24.3%) vs baseline: -1.4% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.4% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.726ms (SLO: <19.150ms 📉 -23.1%) vs baseline: +1.0% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +4.5% ✅ modulo_noaspectTime: ✅ 0.361ms (SLO: <3.000ms 📉 -88.0%) vs baseline: -1.5% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.7% ✅ replace_aspectTime: ✅ 18.403ms (SLO: <24.000ms 📉 -23.3%) vs baseline: +0.3% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.5% ✅ replace_noaspectTime: ✅ 278.934µs (SLO: <300.000µs -7.0%) vs baseline: -1.4% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.8% ✅ repr_aspectTime: ✅ 312.051µs (SLO: <420.000µs 📉 -25.7%) vs baseline: -2.7% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ repr_noaspectTime: ✅ 46.669µs (SLO: <90.000µs 📉 -48.1%) vs baseline: +0.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ rstrip_aspectTime: ✅ 384.689µs (SLO: <500.000µs 📉 -23.1%) vs baseline: +0.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ rstrip_noaspectTime: ✅ 184.152µs (SLO: <300.000µs 📉 -38.6%) vs baseline: +0.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ slice_aspectTime: ✅ 184.628µs (SLO: <300.000µs 📉 -38.5%) vs baseline: +1.0% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ slice_noaspectTime: ✅ 53.892µs (SLO: <90.000µs 📉 -40.1%) vs baseline: -1.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ stringio_aspectTime: ✅ 3.839ms (SLO: <5.000ms 📉 -23.2%) vs baseline: +0.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ stringio_noaspectTime: ✅ 380.814µs (SLO: <500.000µs 📉 -23.8%) vs baseline: +6.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ strip_aspectTime: ✅ 272.896µs (SLO: <350.000µs 📉 -22.0%) vs baseline: -0.6% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ strip_noaspectTime: ✅ 177.101µs (SLO: <240.000µs 📉 -26.2%) vs baseline: -1.0% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ swapcase_aspectTime: ✅ 335.102µs (SLO: <500.000µs 📉 -33.0%) vs baseline: -2.0% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ swapcase_noaspectTime: ✅ 273.099µs (SLO: <400.000µs 📉 -31.7%) vs baseline: -0.4% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.0% ✅ title_aspectTime: ✅ 323.826µs (SLO: <500.000µs 📉 -35.2%) vs baseline: -2.7% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ title_noaspectTime: ✅ 261.015µs (SLO: <400.000µs 📉 -34.7%) vs baseline: -0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ translate_aspectTime: ✅ 492.588µs (SLO: <700.000µs 📉 -29.6%) vs baseline: -1.8% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ translate_noaspectTime: ✅ 423.496µs (SLO: <500.000µs 📉 -15.3%) vs baseline: -2.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ upper_aspectTime: ✅ 295.978µs (SLO: <500.000µs 📉 -40.8%) vs baseline: -2.0% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ upper_noaspectTime: ✅ 238.972µs (SLO: <400.000µs 📉 -40.3%) vs baseline: +1.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 514.252µs (SLO: <700.000µs 📉 -26.5%) vs baseline: 📈 +19.7% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.6% ✅ ospathbasename_noaspectTime: ✅ 430.786µs (SLO: <700.000µs 📉 -38.5%) vs baseline: -1.5% Memory: ✅ 42.487MB (SLO: <46.000MB -7.6%) vs baseline: +4.5% ✅ ospathjoin_aspectTime: ✅ 630.018µs (SLO: <700.000µs -10.0%) vs baseline: +1.4% Memory: ✅ 42.782MB (SLO: <46.000MB -7.0%) vs baseline: +4.6% ✅ ospathjoin_noaspectTime: ✅ 635.218µs (SLO: <700.000µs -9.3%) vs baseline: +0.9% Memory: ✅ 42.625MB (SLO: <46.000MB -7.3%) vs baseline: +4.4% ✅ ospathnormcase_aspectTime: ✅ 347.620µs (SLO: <700.000µs 📉 -50.3%) vs baseline: -0.6% Memory: ✅ 42.507MB (SLO: <46.000MB -7.6%) vs baseline: +4.1% ✅ ospathnormcase_noaspectTime: ✅ 360.485µs (SLO: <700.000µs 📉 -48.5%) vs baseline: ~same Memory: ✅ 42.684MB (SLO: <46.000MB -7.2%) vs baseline: +4.7% ✅ ospathsplit_aspectTime: ✅ 488.011µs (SLO: <700.000µs 📉 -30.3%) vs baseline: -0.7% Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +4.6% ✅ ospathsplit_noaspectTime: ✅ 500.128µs (SLO: <700.000µs 📉 -28.6%) vs baseline: -0.2% Memory: ✅ 42.762MB (SLO: <46.000MB -7.0%) vs baseline: +4.8% ✅ ospathsplitdrive_aspectTime: ✅ 372.176µs (SLO: <700.000µs 📉 -46.8%) vs baseline: +1.0% Memory: ✅ 42.723MB (SLO: <46.000MB -7.1%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 73.066µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.2% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +5.0% ✅ ospathsplitext_aspectTime: ✅ 454.350µs (SLO: <700.000µs 📉 -35.1%) vs baseline: -1.6% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 464.376µs (SLO: <700.000µs 📉 -33.7%) vs baseline: -1.3% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.3% 🟡 Near SLO Breach (1 suite)🟡 tracer - 6/6✅ largeTime: ✅ 31.452ms (SLO: <32.950ms -4.5%) vs baseline: +0.5% Memory: ✅ 36.687MB (SLO: <39.250MB -6.5%) vs baseline: +4.2% ✅ mediumTime: ✅ 3.110ms (SLO: <3.200ms -2.8%) vs baseline: +0.4% Memory: ✅ 35.468MB (SLO: <38.750MB -8.5%) vs baseline: +3.9% ✅ smallTime: ✅ 365.442µs (SLO: <370.000µs 🟡 -1.2%) vs baseline: +4.4% Memory: ✅ 35.665MB (SLO: <38.750MB -8.0%) vs baseline: +4.9%
|
Codeowners resolved as |
004f4fd to
154fff2
Compare
22be021 to
c204828
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
devflow unqueued this merge request: It did not become mergeable within the expected time |
Co-authored-by: Emmett Butler <[email protected]>
Co-authored-by: Emmett Butler <[email protected]>
Description
With DD_CELERY_DISTRIBUTED_TRACING=true, trace headers were not propagated when using app.send_task("remote.task.name", ...) because the task is not in the producer’s local registry. We only injected headers when registry.tasks.get(task_name) returned a task, so we skipped injection for send_task()-by-name. We now inject distributed tracing headers even when the task is not registered locally, so traces link correctly from producer to worker.
Testing
Celery integration tests.
Risks
None.
Additional Notes
N/A
Before
After
Note: The
producer spanis still missing.