Repository navigation
feat(celery): bind worker histories with process_handoffs= and deprecate history= - #161
Conversation
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
worker_contract takes histories= with handoff_gaps= keyed by history name, as RQ's does; history= and gap= remain the single-history form. The reference test adds a chord of two send_message tasks. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Documentation build overview
|
|
2.5s elapsed · 2.2s summed check time Application inventory and skipped tests3 applications with results · 0 applications not run Application inventory was not supplied; uncollected suites cannot be listed. example catalog
View contract setup · Test time: 2.2s summed check time ⚪ Deliberate exclusions
These are scope decisions, not demonstrated defects. Review their reasons when the application changes. Hypothesis · 3/3 scenario checks passedGenerated schedule exploration uses the application's scenario bindings.
These counts are scenario checks, not generated examples. Example counts, shrinking statistics and replay histories are not recorded in this report; inspect the pytest log for a failing history. 🟢 5 supported guarantees
Suite (application code): Noticeboard.publish…
View suite · Test time: 0.0s summed check time Application: Scope: Handoff exemption. Application reason: Publishing commits the complete notice; the optional alert neither finishes nor changes that notice. Hypothesis · Not reportedThis report has no generated exploration evidence. 🟢 1 supported guarantees
Suite (application code): rename_order exempt…
View suite · Test time: 0.0s summed check time Application: Scope: Handoff exemption. Application reason: the read path rebuilds a summary that disagrees with its order, so a lost eviction costs nothing Hypothesis · Not reportedThis report has no generated exploration evidence. 🟢 1 supported guarantees
Suite (application code): What do these numbers mean?
Pass/fail/blocked/rejected counts partition unique checks. Gaps count guarantees, not extra tests. Declined and not-applicable declarations remain separate and do not establish passing coverage. Graphics show zero issues in green; missing reports never show green zeroes. Elapsed time covers the pytest session. Summed check time includes setup, execution and cleanup; parallel work overlaps. Application names, quoted reasons and diagnostic values come from application code. Guarantee labels come from pytest-obligation. Results cover collected suites only; passing checks do not rate every test's quality. Test step (GitHub): success |
|
Runners: Selection and execution
|
| Lane | Because | |
|---|---|---|
core |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
minimum_models |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
exploration |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
django |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
django_psycopg2 |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
redis |
selected | affected test package/tests/celery_worker/test_celery_worker_contract.pyaffected test package/tests/celery_worker/test_celery_worker_contract_arguments.pyaffected test package/tests/celery_worker/test_celery_worker_platform.py+2 more |
mongodb |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
aiokafka |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
prefect |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
demos |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
saleor |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
actions |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
due |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
typecheck |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.pyexplicit input package/tests/celery_worker/app.pyexplicit input package/tests/celery_worker/test_celery_worker_contract.py+1 more |
contracts |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.pyexplicit input package/tests/celery_worker/app.pyexplicit input package/tests/celery_worker/test_celery_worker_contract.py+1 more |
wheel |
selected | explicit input package/src/pytest_obligation/integrations/celery_worker.py |
What else avoided work?
unavailable reused · 13/13 environment cache hits
Reused validation evidence
Reuse evidence was not recorded. Skipped jobs alone do not prove reuse.
Environment caching
| Job | Observed outcome after locked sync | Validation / sync |
|---|---|---|
aiokafka |
hit | 2s |
django-psycopg2 |
hit | 1s |
django |
hit | 1s |
redis |
hit | 1s |
exploration |
hit | 1s |
core |
hit | 1s |
core |
hit | 1s |
minimum-models |
hit | 1s |
prefect |
hit | 1s |
mongodb |
hit | 1s |
core |
hit | 1s |
core |
hit | 1s |
lint |
hit | 0s |
Every recorded environment passed locked synchronization. A restored but unusable environment is recorded as repaired. Download caches are separate and are not included in these counts.
Cache time saved is unavailable without a comparable cold run.
Incremental test evidence is not inferred from skipped jobs or added to whole-lane reuse totals.
CI time and cost
4m 20s merge-ready · 48 min runner · $0.00 est. cost
Runner minutes use per-job rounding; cost is an estimate.
| Workflow | Merge-ready | First runner | Longest job | Runner time | Cost |
|---|---|---|---|---|---|
| CI | 4m 20s | after 2s | saleor · 3m 58s | 48 min | $0 |
Timeline and critical path
%%{init: {"theme": "base", "themeVariables": {"fontFamily": "-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif", "fontSize": "13px", "doneTaskBkgColor": "#3B82F6", "doneTaskBorderColor": "#2563EB", "critBkgColor": "#EA580C", "activeTaskBkgColor": "#2EA043", "activeTaskBorderColor": "#1A7F37", "critBorderColor": "#C2410C", "taskBkgColor": "#3B82F6", "taskBorderColor": "#2563EB", "taskTextColor": "#FFFFFF", "taskTextDarkColor": "#FFFFFF", "taskTextLightColor": "#FFFFFF", "taskTextOutsideColor": "#8B949E", "titleColor": "#8B949E", "sectionBkgColor": "rgba(234, 88, 12, 0.07)", "altSectionBkgColor": "rgba(139, 148, 158, 0.05)", "sectionBkgColor2": "rgba(234, 88, 12, 0.07)", "gridColor": "rgba(139, 148, 158, 0.3)", "todayLineColor": "rgba(0, 0, 0, 0)"}, "gantt": {"barHeight": 20, "barGap": 6, "topPadding": 44, "leftPadding": 230, "sectionFontSize": 13, "fontSize": 12}}}%%
gantt
title 4m 20s to merge-ready · orange is what the merge waited on
dateFormat YYYY-MM-DD HH:mm:ss
axisFormat %M:%S
todayMarker off
section lint
12s :done, j0, 2000-01-01 00:00:02, 2000-01-01 00:00:14
section select-tests
13s :crit, j1, 2000-01-01 00:00:02, 2000-01-01 00:00:15
section actions
38s :done, j2, 2000-01-01 00:00:17, 2000-01-01 00:00:55
section aiokafka
45s :done, j3, 2000-01-01 00:00:17, 2000-01-01 00:01:02
section build
16s :done, j4, 2000-01-01 00:00:17, 2000-01-01 00:00:33
section ci-contracts
26s :done, j5, 2000-01-01 00:00:17, 2000-01-01 00:00:43
section core · 3.11
2m 1s :done, j6, 2000-01-01 00:00:17, 2000-01-01 00:02:18
section core · 3.12
3m 21s :done, j7, 2000-01-01 00:00:17, 2000-01-01 00:03:38
section demos
3m 56s :done, j8, 2000-01-01 00:00:17, 2000-01-01 00:04:13
section django
1m 8s :done, j9, 2000-01-01 00:00:17, 2000-01-01 00:01:25
section django-psycopg2
1m 2s :done, j10, 2000-01-01 00:00:17, 2000-01-01 00:01:19
section due
2m 54s :done, j11, 2000-01-01 00:00:17, 2000-01-01 00:03:11
section minimum-models
2m 26s :done, j12, 2000-01-01 00:00:17, 2000-01-01 00:02:43
section mongodb
35s :done, j13, 2000-01-01 00:00:17, 2000-01-01 00:00:52
section prefect
1m 48s :done, j14, 2000-01-01 00:00:17, 2000-01-01 00:02:05
section redis
2m 31s :done, j15, 2000-01-01 00:00:17, 2000-01-01 00:02:48
section saleor
3m 58s :crit, j16, 2000-01-01 00:00:17, 2000-01-01 00:04:15
section typecheck
16s :done, j17, 2000-01-01 00:00:17, 2000-01-01 00:00:33
section core · 3.13
2m 18s :done, j18, 2000-01-01 00:00:18, 2000-01-01 00:02:36
section core · 3.14
2m 22s :done, j19, 2000-01-01 00:00:18, 2000-01-01 00:02:40
section exploration
2m 37s :done, j20, 2000-01-01 00:00:18, 2000-01-01 00:02:55
section Validation gate
3s :crit, j21, 2000-01-01 00:04:17, 2000-01-01 00:04:20
section ✅ merge-ready 4m 20s
✅ :active, m, 2000-01-01 00:04:20, 2000-01-01 00:05:38
⏳: job creation-to-start delay, including scheduling/admission. Orange: the critical path through the workflow's needs graph. Bars are observed API timestamps.
Critical path
Observed dependency chain (4m 20s): select-tests → saleor → Validation gate
Execution-only CPM (4m 14s): select-tests → saleor → Validation gate
| Job | Execution (s) | Delay after prerequisites (s) | Execution slack (s) |
|---|---|---|---|
| select-tests | 13 | 2 | 0 |
| lint | 12 | 2 | 239 |
| ci-contracts | 26 | 2 | 212 |
| typecheck | 16 | 2 | 222 |
| core (3.11) | 121 | 2 | 117 |
| core (3.12) | 201 | 2 | 37 |
| core (3.13) | 138 | 3 | 100 |
| core (3.14) | 142 | 3 | 96 |
| minimum-models | 146 | 2 | 92 |
| exploration | 157 | 3 | 81 |
| django | 68 | 2 | 170 |
| django-psycopg2 | 62 | 2 | 176 |
| redis | 151 | 2 | 87 |
| mongodb | 35 | 2 | 203 |
| aiokafka | 45 | 2 | 193 |
| prefect | 108 | 2 | 130 |
| demos | 236 | 2 | 2 |
| saleor | 238 | 2 | 0 |
| build | 16 | 2 | 222 |
| actions | 38 | 2 | 200 |
| due | 174 | 2 | 64 |
| Validation gate | 3 | 2 | 0 |
Delay includes scheduling, runner admission and other waits; it is not isolated queue time. Execution CPM holds measured durations constant and excludes those delays. The declared DAG must cover every prerequisite; this analysis does not assert validation success.
Runner minutes and cost by runner and job
Estimated runner usage for this run: 48 rounded min from 35.8 raw min; 12.2 min (25%) is per-job rounding.
Estimated runner cost at recorded rates: $0.
| runner SKU | rounded min | USD/min | estimated cost |
|---|---|---|---|
| github_public_actions_linux | 48 | $0 | $0 |
Hosted runner rates: GitHub pricing, verified 2026-10-24. List-price estimates are before included minutes, discounts and taxes; they are not an invoice. Known public repositories use the free standard-runner rate; larger runners remain billable. Explicit configured rates take precedence.
| job | runner labels | runner / group | attempt | wall | rounded min | save 1 min | headroom |
|---|---|---|---|---|---|---|---|
| saleor | ubuntu-latest |
GitHub Actions 1000528395 / GitHub Actions |
1 | 238s | 4 | 58s | 2s |
| demos | ubuntu-latest |
GitHub Actions 1000528396 / GitHub Actions |
1 | 236s | 4 | 56s | 4s |
| core (3.12) | ubuntu-latest |
GitHub Actions 1000528394 / GitHub Actions |
1 | 201s | 4 | 21s | 39s |
| due | ubuntu-latest |
GitHub Actions 1000528386 / GitHub Actions |
1 | 174s | 3 | 54s | 6s |
| exploration | ubuntu-latest |
GitHub Actions 1000528381 / GitHub Actions |
1 | 157s | 3 | 37s | 23s |
| redis | ubuntu-latest |
GitHub Actions 1000528389 / GitHub Actions |
1 | 151s | 3 | 31s | 29s |
| minimum-models | ubuntu-latest |
GitHub Actions 1000528385 / GitHub Actions |
1 | 146s | 3 | 26s | 34s |
| core (3.14) | ubuntu-latest |
GitHub Actions 1000528387 / GitHub Actions |
1 | 142s | 3 | 22s | 38s |
| core (3.13) | ubuntu-latest |
GitHub Actions 1000528383 / GitHub Actions |
1 | 138s | 3 | 18s | 42s |
| core (3.11) | ubuntu-latest |
GitHub Actions 1000528393 / GitHub Actions |
1 | 121s | 3 | 1s | 59s |
| prefect | ubuntu-latest |
GitHub Actions 1000528391 / GitHub Actions |
1 | 108s | 2 | 48s | 12s |
| django | ubuntu-latest |
GitHub Actions 1000528382 / GitHub Actions |
1 | 68s | 2 | 8s | 52s |
| django-psycopg2 | ubuntu-latest |
GitHub Actions 1000528380 / GitHub Actions |
1 | 62s | 2 | 2s | 58s |
| aiokafka | ubuntu-latest |
GitHub Actions 1000528378 / GitHub Actions |
1 | 45s | 1 | remove/consolidate | 15s |
| actions | ubuntu-latest |
GitHub Actions 1000528390 / GitHub Actions |
1 | 38s | 1 | remove/consolidate | 22s |
| mongodb | ubuntu-latest |
GitHub Actions 1000528392 / GitHub Actions |
1 | 35s | 1 | remove/consolidate | 25s |
| ci-contracts | ubuntu-latest |
GitHub Actions 1000528384 / GitHub Actions |
1 | 26s | 1 | remove/consolidate | 34s |
| typecheck | ubuntu-latest |
GitHub Actions 1000528388 / GitHub Actions |
1 | 16s | 1 | remove/consolidate | 44s |
| build | ubuntu-latest |
GitHub Actions 1000528379 / GitHub Actions |
1 | 16s | 1 | remove/consolidate | 44s |
| select-tests | ubuntu-latest |
GitHub Actions 1000528376 / GitHub Actions |
1 | 13s | 1 | remove/consolidate | 47s |
| lint | ubuntu-latest |
GitHub Actions 1000528374 / GitHub Actions |
1 | 12s | 1 | remove/consolidate | 48s |
| Validation gate | ubuntu-latest |
GitHub Actions 1000528458 / GitHub Actions |
1 | 3s | 1 | remove/consolidate | 57s |
The organization billing ledger is authoritative; this estimate applies a per-job minute ceiling to GitHub timestamps.
Slowest step in each job
| job | result | job time | slowest timed step | step time |
|---|---|---|---|---|
saleor |
success | 238s (complete) | Saleor checkout |
181s (complete) |
demos |
success | 236s (complete) | Wagtail on django-tasks-db |
74s (complete) |
core (3.12) |
success | 201s (complete) | Prove package-only core controls need no application checkout |
90s (complete) |
due |
success | 174s (complete) | Run complete work-table and application conformance |
93s (complete) |
exploration |
success | 157s (complete) | Run uv run --no-sync pytest package/tests/core integration_tests/core |
142s (complete) |
redis |
success | 151s (complete) | Run uv run --no-sync pytest package/tests/rq package/tests/celery_worker integration_tests/rq integration_tests/celery_worker -p no:django -rxX -n 4 --durations=10 --due-work-explore=smoke --due-work-check-fixtures --due-work-require-assessed --due-work-profile-report=profiles-redis.json --junitxml=adopters-redis.xml |
128s (complete) |
minimum-models |
success | 146s (complete) | Run uv run --no-sync pytest package/tests/core integration_tests/core |
126s (complete) |
core (3.14) |
success | 142s (complete) | Run uv run --python 3.14 --no-sync pytest package/tests/core integration_tests/core --junitxml=test-timings.xml -o junit_family=xunit1 |
128s (complete) |
core (3.13) |
success | 138s (complete) | Run uv run --python 3.13 --no-sync pytest package/tests/core integration_tests/core --junitxml=test-timings.xml -o junit_family=xunit1 |
117s (complete) |
core (3.11) |
success | 121s (complete) | Run uv run --python 3.11 --no-sync pytest package/tests/core integration_tests/core --junitxml=test-timings.xml -o junit_family=xunit1 |
105s (complete) |
prefect |
success | 108s (complete) | Run uv run --no-sync pytest package/tests/prefect integration_tests/prefect -q --due-work-explore=smoke --due-work-check-fixtures --due-work-require-assessed --due-work-profile-report=profiles-prefect.json --junitxml=adopters-prefect.xml --durations=20 |
96s (complete) |
django |
success | 68s (complete) | Run uv run --no-sync pytest package/tests/django examples/adopter/tests_django --ds=tests.django.settings -n 4 |
38s (complete) |
django-psycopg2 |
success | 62s (complete) | Run uv run --no-sync pytest package/tests/django examples/adopter/tests_django --ds=tests.django.settings -n 4 |
29s (complete) |
aiokafka |
success | 45s (complete) | Initialize containers |
22s (complete) |
actions |
success | 38s (complete) | Generated histories also support optional search |
6s (complete) |
mongodb |
success | 35s (complete) | Run uv run --no-sync pytest package/tests/mongodb -q |
13s (complete) |
ci-contracts |
success | 26s (complete) | Prove consumer routing and documentation checks |
16s (complete) |
typecheck |
success | 16s (complete) | Run uv sync --all-packages --locked --all-extras --group demos |
3s (complete) |
build |
success | 16s (complete) | Verify old and new adopters against the installed wheel |
3s (complete) |
select-tests |
success | 13s (complete) | Run gigaverse-app/pytest-selective/actions/plan-lanes@v1 |
5s (complete) |
lint |
success | 12s (complete) | Run gigaverse-app/ci-esta/actions/setup-cached-uv@v1 |
4s (complete) |
Validation gate |
success | 3s (complete) | Set up job |
1s (complete) |
Latest attempt only. All observed steps are retained in the JSON report. Unavailable timestamps are not zero; accrued time is measured at the snapshot. Step times explain job time and are not added to runner usage.
Savings across selection, reuse, and caching are not additive.
CI-esta · Let proven work rest. Measure the rest.
Reading this report · run 40179895071 · schema ci-observability/v1
…histories # Conflicts: # package/tests/celery_worker/test_celery_worker_contract.py
…act's design checks ObligationContract already refuses duplicate handoff history names and handoff_gaps that name no history; worker_contract no longer repeats them. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…tory= worker_contract takes the adopter's histories as ObligationContract does, process_handoffs= with handoff_gaps= by history name, replacing the unreleased histories=. history= and gap= still bind and now warn. The reference test builds both histories through one helper, and gates the rows the #10724 fixes remove through one helper with its retirement condition. The docs describe the binding beside the integrations table. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ate history= (#161) ## TLDR Today `celery_worker.worker_contract()` binds exactly one history, so an adopter checking a task and a chord has to `model_copy` the second history onto `process_handoffs` and `handoff_gaps` by hand. celery/celery#10726 does exactly that. This PR makes `worker_contract` take its histories the way `ObligationContract` itself does: `process_handoffs=` with `handoff_gaps=` keyed by history name, the same field names with the same meaning, so nothing is translated between two vocabularies. The one-history form, `history=` with `gap=`, is deprecated: it still binds, and emits a `DeprecationWarning` naming the replacement. A mix of the two forms is refused in words. The reference self-test now binds a chord of two `send_message` tasks next to the single task. On the locked Celery 5.6.3, that chord reproduces celery/celery#10724's chord case on a real worker: a header lost after its SUCCESS was stored, or whose `on_success` raised, fails the whole chord. Builds on #160 (merged), which gates the fixed rows on the Celery release. ```text BEFORE AFTER worker_contract(history=TASK, gap=...) worker_contract(process_handoffs=(TASK,), handoff_gaps={TASK.name: ...}) (history=/gap= still work and warn: DeprecationWarning) a second history: model_copy(update={ worker_contract(process_handoffs=(TASK, CHORD), "process_handoffs": (..., CHORD), handoff_gaps={TASK.name: ..., CHORD.name: ...}) "handoff_gaps": {..., CHORD.name: ...}}) ``` ## What the chord history pins The faults hit the first execution of a header task. The refused publish is the last header publishing the body. | History | Celery ≤ 5.7.0rc1 (lock 5.6.3) | Later releases (verified on Celery `main`) | | --- | --- | --- | | normal operation | SUCCESS, 2 sent, `sent 2 messages` | same | | died at `task_prerun` | FAILURE, 1 sent, `failed: ChordError` | same | | died at `mark_as_done` | FAILURE, 2 sent, `failed: ChordError` (celery/celery#10949) | same | | died at `task_postrun` | FAILURE, 2 sent, `failed: ChordError` (celery/celery#10724) | normal operation | | `on_success` raised | FAILURE, 2 sent, `failed: ChordError` (celery/celery#10724) | normal operation | | broker refused the body | FAILURE, 2 sent, `failed: ChordError` ([#10725 comment](celery/celery#10725 (comment))) | same | ## Refused bindings `worker_contract` asserts, naming the contract: - not both the deprecated `history=` and `process_handoffs=` or `handoff_gaps=`; - no `gap=` without `history=`; - at least one history. Two histories sharing a name, and a `handoff_gaps` key naming no bound history, are refused by `ObligationContract`'s own design checks (`ObligationContractDesignError`), so `worker_contract` doesn't repeat them; a test pins that they apply to several histories. ## Reference test and docs - Both histories are built by one `reference_history(name, send, findings)` helper instead of two near-identical `worker_history` calls. - The rows #10752/#10759 remove are added through one `before_the_10724_fixes(rows)` helper beside the gate, which now states its retirement condition: delete it and those rows once `uv.lock`'s Celery is newer than 5.7.0rc1. - `package/docs/integrations.md` describes the binding beside the integrations table, with an example and the deprecation. ## Tested WSL, Python 3.12, Redis 7.4.11, `package/tests/celery_worker -p no:django -rxX`: | Celery | Result | | --- | --- | | 5.6.3 (the lock) | 15 passed, 3 xfailed | | `main` at e3cc7cc53, release gate forced on | 15 passed, 3 xfailed | - `test_celery_worker_contract_arguments.py`: 8 passed (the binding, the deprecated form binding with its warning, 4 mixed-form refusals, 2 core design refusals). - After these changes, `package/tests/celery_worker` on Celery 5.6.3 with real workers: 23 passed, 3 xfailed. It needs no worker, so it also runs on Windows. - `ruff check package` and `ruff format --check package`: clean. - `pyrefly check`: no errors in `celery_worker` files. The 5.6.3 run shows 7 `DeprecationWarning`s from Celery's own Redis backend (`setex`) on the chord path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
The fault commits were written over an older copy of #161 and reverted its process_handoffs= binding, the history= deprecation and the move of name checks into the contract. This restores them with a three-way merge and binds the SIGKILL and full-broker histories through reference_history. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…broker (#165) ## TLDR Today the Celery worker integration fakes the two failures it is about. A "death" is the pool child calling `os._exit(1)` from inside a hook, and a "refused link" is a patched `Signature.apply_async` raising kombu's `OperationalError`. Both are realistic: a probe against celery/celery#10726 got identical outcomes from a real outside `kill -9` and from a real Redis broker full under `noeviction`. But that probe was a one-off script, and Celery reports a signal death through a different path from an exit status. This PR makes the real failures part of the integration, so every contract can run them: - **`SIGKILL_DEATH_POINTS`:** the kernel kills the pool child at the same three stages. - **`BROKER_FULL`:** the broker's own Redis server refuses writes once the task's body has run. Both are opt-in, so no adopter's pinned findings table changes. The reference contract runs both, on Celery 5.6.3 and on Celery `main`. **Stacked on #161** (and #160). Review the last three commits. ## The fault list ```text BEFORE AFTER deaths (os._exit(1) in the pool child) deaths (os._exit(1)), default, unchanged task_prerun, mark_as_done, task_postrun task_prerun, mark_as_done, task_postrun SIGKILL_DEATH_POINTS (kernel SIGKILL), opt-in task_prerun (SIGKILL), mark_as_done (SIGKILL), task_postrun (SIGKILL) failures the worker survives, default failures the worker survives, default, unchanged the task's on_success hook raised the task's on_success hook raised the broker refused the task's link the broker refused the task's link (patched apply_async) (patched apply_async) BROKER_FULL, opt-in, needs broker_url= the Redis broker was full (CONFIG SET maxmemory 1, noeviction) ``` ## Why opt-in Every adopter's `Findings` table pins every history it runs. A new default death point would show up in their tables as an unpinned history and turn their suites red on upgrade. With opt-in, an adopter adds them when it chooses: ```python worker_history(..., death_points=DEATH_POINTS + SIGKILL_DEATH_POINTS) worker_history(..., death_points=(), failure_points=(BROKER_FULL,), broker_url=OWN_BROKER) ``` `BROKER_FULL` also needs a broker on a Redis server shared with nothing else: `maxmemory` is server-wide, so the result backend and anything the test reads must live elsewhere. ## How the broker-full fault is contained - **In the worker's pool child:** after the task's body returns, on its first execution only, the child sets the broker's server to `maxmemory 1` with `noeviction`. Redis then refuses the writes that follow, and kombu re-raises Redis's `OutOfMemoryError` as `OperationalError`. - **In the test process:** `broker_restored(broker_url)` reads `maxmemory` and `maxmemory-policy` before the run and restores them on exit, before the worker stops and before recovery. That happens whether the run settled, failed or timed out. - **Refused in words:** a missing or non-Redis `broker_url` at declaration, a non-Redis broker in the worker, and a server that refuses `CONFIG`. ## What the reference contract pins WSL, Python 3.12, Redis 7.4. The broker was a separate Redis container. `package/tests/celery_worker -p no:django -rxX`. | History | Celery 5.6.3 (the lock) | Celery `main` e3cc7cc53 (gate forced on) | | --- | --- | --- | | died at `task_prerun (SIGKILL)` | FAILURE, 0 sent, `failed: WorkerLostError` | same | | died at `mark_as_done (SIGKILL)` | FAILURE, 1 sent, `failed: WorkerLostError` + `sent` (celery/celery#10949) | same | | died at `task_postrun (SIGKILL)` | SUCCESS, 1 sent, `failed: WorkerLostError` + `sent` (celery/celery#10724) | normal operation (fixed by #10752) | | the Redis broker was full | FAILURE, 1 sent, `failed: OperationalError`, link never sent (celery/celery#10725) | same | Each SIGKILL row matches its `os._exit` row on the same Celery version. The full broker matches the injected refusal row. So the injected faults were faithful, and the suite will notice if Celery's signal path or its refusal handling ever diverges from them. | Run | Result | | --- | --- | | Celery 5.6.3 | 23 passed, 4 xfailed; broker server's `maxmemory` back to `0` afterwards | | Celery `main`, gate forced on | 23 passed, 4 xfailed | | `test_celery_worker_broker_full.py` | 3 passed: refusals in words, and the restore when the run raises | - `ruff check`, `ruff format --check`: clean. - `pyrefly check`: no errors in Celery files. ## CI The `redis` job gains a second service, `redis-own-broker` on port 6380, and `CELERY_OWN_BROKER_URL`. Only the broker-full history uses it, so filling it cannot refuse writes for the other xdist workers' suites. `test_the_broker_is_restored_however_the_run_ends` changes the shared server's `maxmemory` only to 512 MiB, never to a value that refuses writes. **Stacked on #161 and built on its API.** The first commits were written over an older copy of #161 and reverted its `process_handoffs=` binding and `history=` deprecation; fb33765 restores them by three-way merge. Since then the new histories are bound through #161's `reference_history` and `process_handoffs=`. Relative to #161 this PR now only adds: 26 passed, 4 xfailed locally on Celery 5.6.3, with no `history=` deprecation warning raised. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This is the manual release decision. Leave this PR open to accumulate alpha changes without publishing. **CI approval only runs tests; it does not publish. Merging this PR authorizes publication:** after normal CI passes on the merged commit, the Release workflow tags, builds and publishes that exact version to PyPI. If GitHub asks to approve workflows on each update, the release bot is using GITHUB_TOKEN. The configured GV_CI_CROSS_REPO_TOKEN avoids that prompt; see RELEASING.md. --- ## [0.20.0](v0.19.2...v0.20.0) (2026-10-25) ### Features * **celery:** bind worker histories with process_handoffs= and deprecate history= ([#161](#161)) ([d2b8ac2](d2b8ac2)) * **report:** publish fork PR evidence with an isolated renderer ([#162](#162)) ([6936ca6](6936ca6)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
…broker (land #165 on main) (#172) ## TLDR #165 (SIGKILL deaths and a real full Redis broker for the Celery worker integration) shows as merged, but its base was #161's branch, and it merged there 17 minutes after #161 itself had merged to `main`. Its change never reached `main`, so 0.20.0 has neither `SIGKILL_DEATH_POINTS` nor `BROKER_FULL`. This PR carries #165's diff (its merge commit f8a7b50 against its first parent) onto `main` unchanged. ```text BEFORE #161 ──merged──> main ──> 0.20.0 #165 ──merged──> claude/celery-worker-histories (dead branch) AFTER #161 + this PR ──> main ──> next release (SIGKILL_DEATH_POINTS, BROKER_FULL, broker_restored) ``` What it adds is described in #165: all opt-in, so no adopter's findings table moves. ## Tested On this branch, in WSL with Python 3.12, Redis 7.4.11 and a second Redis for the full-broker history, `package/tests/celery_worker`: 26 passed, 4 xfailed. `ruff check` and `ruff format --check`: clean. Found while building a chord contract on gigaverse-app/celery: `from pytest_obligation.integrations.celery_worker import SIGKILL_DEATH_POINTS` fails on 0.20.0. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <[email protected]>
…broker (land #165 on main) (#172) ## TLDR #165 (SIGKILL deaths and a real full Redis broker for the Celery worker integration) shows as merged, but its base was #161's branch, and it merged there 17 minutes after #161 itself had merged to `main`. Its change never reached `main`, so 0.20.0 has neither `SIGKILL_DEATH_POINTS` nor `BROKER_FULL`. This PR carries #165's diff (its merge commit f8a7b50 against its first parent) onto `main` unchanged. ```text BEFORE #161 ──merged──> main ──> 0.20.0 #165 ──merged──> claude/celery-worker-histories (dead branch) AFTER #161 + this PR ──> main ──> next release (SIGKILL_DEATH_POINTS, BROKER_FULL, broker_restored) ``` What it adds is described in #165: all opt-in, so no adopter's findings table moves. ## Tested On this branch, in WSL with Python 3.12, Redis 7.4.11 and a second Redis for the full-broker history, `package/tests/celery_worker`: 26 passed, 4 xfailed. `ruff check` and `ruff format --check`: clean. Found while building a chord contract on gigaverse-app/celery: `from pytest_obligation.integrations.celery_worker import SIGKILL_DEATH_POINTS` fails on 0.20.0. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <[email protected]>
TLDR
Today
celery_worker.worker_contract()binds exactly one history, so an adopter checking a task and a chord has tomodel_copythe second history ontoprocess_handoffsandhandoff_gapsby hand. celery/celery#10726 does exactly that. This PR makesworker_contracttake its histories the wayObligationContractitself does:process_handoffs=withhandoff_gaps=keyed by history name, the same field names with the same meaning, so nothing is translated between two vocabularies. The one-history form,history=withgap=, is deprecated: it still binds, and emits aDeprecationWarningnaming the replacement. A mix of the two forms is refused in words.The reference self-test now binds a chord of two
send_messagetasks next to the single task. On the locked Celery 5.6.3, that chord reproduces celery/celery#10724's chord case on a real worker: a header lost after its SUCCESS was stored, or whoseon_successraised, fails the whole chord.Builds on #160 (merged), which gates the fixed rows on the Celery release.
What the chord history pins
The faults hit the first execution of a header task. The refused publish is the last header publishing the body.
main)sent 2 messagestask_prerunfailed: ChordErrormark_as_donefailed: ChordError(celery/celery#10949)task_postrunfailed: ChordError(celery/celery#10724)on_successraisedfailed: ChordError(celery/celery#10724)failed: ChordError(#10725 comment)Refused bindings
worker_contractasserts, naming the contract:history=andprocess_handoffs=orhandoff_gaps=;gap=withouthistory=;Two histories sharing a name, and a
handoff_gapskey naming no bound history, are refused byObligationContract's own design checks (ObligationContractDesignError), soworker_contractdoesn't repeat them; a test pins that they apply to several histories.Reference test and docs
reference_history(name, send, findings)helper instead of two near-identicalworker_historycalls.before_the_10724_fixes(rows)helper beside the gate, which now states its retirement condition: delete it and those rows onceuv.lock's Celery is newer than 5.7.0rc1.package/docs/integrations.mddescribes the binding beside the integrations table, with an example and the deprecation.Tested
WSL, Python 3.12, Redis 7.4.11,
package/tests/celery_worker -p no:django -rxX:mainat e3cc7cc53, release gate forced ontest_celery_worker_contract_arguments.py: 8 passed (the binding, the deprecated form binding with its warning, 4 mixed-form refusals, 2 core design refusals).package/tests/celery_workeron Celery 5.6.3 with real workers: 23 passed, 3 xfailed. It needs no worker, so it also runs on Windows.ruff check packageandruff format --check package: clean.pyrefly check: no errors incelery_workerfiles.The 5.6.3 run shows 7
DeprecationWarnings from Celery's own Redis backend (setex) on the chord path.🤖 Generated with Claude Code