Skip to content

Build the abi3 wheel once per platform, test it per interpreter, warm the caches and add Python 3.15 - #684

Merged
evertlammerts merged 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:ci-build-once-test-across
Oct 16, 2026
Merged

evertlammerts merged 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:ci-build-once-test-across

Conversation

@evertlammerts

Copy link
Copy Markdown
Member

While checking what Python 3.15 support would take, I looked at what a pull request run on this branch actually does. Run 38863090412 (2026-10-16) took 63 minutes and compiled everything cold in all 13 heavy jobs: the Linux seed logged "No cache found" and 0 of 526 ccache hits. The compiler cache was never warmed on this branch. The cache-warm trigger and the save condition in the ccache action list main, v1.4-andium and v1.5-variegata, a list written in February and never updated when v2.0-cyanoptera branched off in September. The caches API had no ccache entry for any ref at all.

The run also tested the same thing repeatedly. The cp312 abi3 wheel is one binary per platform, and it ran the full suite inside cibuildwheel on 3.12, then again from the artifact on 3.13 and 3.14, and the sdist job compiled the same source a fourth time and ran the suite once more. Meanwhile no job in a ready PR ran with assertions: every wheel is Release with NDEBUG, so D_ASSERT and the GIL asserts in the bindings were dead in CI. Drafts built the coverage build twice per push, once from on-push and once from on-PR. Windows and macOS saw only 3.12.

This PR restructures the workflows around a few rules, one per event.

The cp312 abi3 wheel is built once per platform by the seed jobs, which run no inline tests. It is then installed from the artifact into every interpreter it serves and the suite runs there: 3.12 through 3.15 on Linux, the oldest and newest of those on Windows and macOS in pull request runs, all of them in release runs. The cp311 wheel keeps its inline cibuildwheel test, since it serves only the interpreter that built it. The sdist job builds, installs, smoke-imports and checks versions in minimal mode and leaves the suite to the wheel jobs; full runs still test it.

Caches are written only on protected branches. The save condition is github.ref_protected instead of a branch list, so the next release branch gets caches the day it gets a ruleset. The only writer is the cache-warm workflow on a push to such a branch: the seed wheels, the new Debug build, and the pre-commit cache. Release runs no longer save, because they may build at an engine commit other than the pin and would replace the pin's cache. The setup-uv caches are off everywhere; they were saving 466 to 923 MB per ref and had pushed the ccaches out of the 10 GB budget.

A new workflow, assert_test.yml, builds the editable full-API Debug install on Linux, the one build with assertions live, and runs the fast suite. It runs for ready PRs and in the merge queue, with identical inputs so a queue failure is never a surprise, and on every push to a fork branch through on-push. Drafts run code quality only. Coverage is dispatch only.

Python 3.15 comes along for free on the wheel side: the abi3 wheel built from this branch against 3.15.0rc3 passed the fast suite here (6392 passed, 26 skipped, 80 xfailed). The matrix gains 3.15, the classifier is added, adbc-driver-manager is excluded on 3.15 because 1.12.0 has no cp315 wheels (its tests importorskip), and three pandas tests stop using rename(inplace=True), which pandas 3.1 deprecates with a warning our pytest config turns into an error. Per-platform resolution of the test group for 3.15 was checked with uv pip compile; torch resolves everywhere except macOS x86_64, which the existing marker already excluded. Tool versions move to setup-uv v10.3.0, uv 0.12.23 and cibuildwheel 4.3.1. Since setup-uv v6 no longer activates the venv, the codspeed workflow now asks for that explicitly; without it the benchmark would run /bin/python under valgrind.

Trade-offs and what is not done. The first PR run after this merges still pays cold builds, because nothing has warmed the caches on this branch yet; the push that merges this does. The sdist job compiles without ccache and is now the slowest uncached Linux leg; I want to see its time before adding a cache to a build that runs in uv's isolated temporary directory. The Debug ccache size is unknown and may approach the 1500 MB cap. The assert job runs on 3.12 only, so the 3.13-and-later branch of the GIL check is never compiled with assertions; a second leg is cheap to add if we see a reason. The editable sync in the assert job uses --no-build-isolation-package duckdb rather than the global flag, because the global flag also hits pyspark, an sdist that needs setuptools, which only works on machines whose uv cache already holds a built pyspark wheel.

Reviewed by Codex and by two Claude rounds plus the code-review skill, about thirty findings in total, all either applied or recorded. actionlint is clean apart from four pre-existing notes about required inputs that also carry a default.

@evertlammerts
evertlammerts force-pushed the ci-build-once-test-across branch from 5638a1c to acf1558 Compare October 16, 2026 11:58
@evertlammerts
evertlammerts merged commit 02a7c08 into duckdb:v2.0-cyanoptera Oct 16, 2026
29 of 31 checks passed
@evertlammerts
evertlammerts deleted the ci-build-once-test-across branch October 16, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant