Skip to content

🔨 Run tests with pytest-xdist and pytest-cov#14992

Merged
tiangolo merged 13 commits intomasterfrom
xdist
Feb 27, 2026
Merged

🔨 Run tests with pytest-xdist and pytest-cov#14992
tiangolo merged 13 commits intomasterfrom
xdist

Conversation

@YuriiMotov
Copy link
Member

@YuriiMotov YuriiMotov commented Feb 24, 2026

Tests run x2.3 faster (locally): 18s VS 42s.

The drawback is that it's easy to introduce race condition. We need to be careful and mark tests that modify files in working directory with @workdir_lock decorator introduced in this PR.

To make it easier to debug race conditions, I added pytest-timeout plugin.

Stress-tested with -n 50 - adjusted pytest timeout as a result (5 -> 20).
After all adjustments, 10 runs passed ✅


Note: I added test-cov.sh script and moved --cov --cov-context=test to it. This way when we run just test.sh it works much faster (doesn't store coverage data).


Inline snapshot gives the following:

INFO: inline-snapshot was disabled because you used xdist. This means that tests with snapshots will continue to run, but snapshot(x) will only return x and inline-snapshot will not be able to fix snapshots or generate reports.

To update inline snapshots we can run tests as bash scripts/test.sh -n 0

@codspeed-hq
Copy link

codspeed-hq bot commented Feb 24, 2026

Merging this PR will not alter performance

✅ 20 untouched benchmarks


Comparing xdist (a754bc1) with master (daba0aa)

Open in CodSpeed

@github-actions
Copy link
Contributor

github-actions bot commented Feb 24, 2026

@YuriiMotov YuriiMotov force-pushed the xdist branch 2 times, most recently from fe938ee to 7365d6f Compare February 24, 2026 18:13
@YuriiMotov YuriiMotov marked this pull request as draft February 24, 2026 18:15
@YuriiMotov YuriiMotov force-pushed the xdist branch 4 times, most recently from 98e1855 to 7b4460b Compare February 24, 2026 20:33
@YuriiMotov YuriiMotov marked this pull request as ready for review February 25, 2026 10:22
Copy link
Member

@tiangolo tiangolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks a lot, great and thorough job. 🙌

Testing everything locally for me went form 2m15s to 1m37s 🚀

@tiangolo tiangolo merged commit 5aacc7b into master Feb 27, 2026
40 checks passed
@tiangolo tiangolo deleted the xdist branch February 27, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants