Repository navigation
fix: update the data file hash when a parallel data file is saved again - #2339
Merged
nedbat merged 1 commit intoOct 21, 2026
Merged
Conversation
A parallel data file got the hash of its data in its name on the first save, and later saves added data without changing the name. Combining could then skip the file as a duplicate of another one that had the same data at the first save, and delete it.
Member
|
And thanks for the quick fix! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2338. Thanks @nedbat for the quick go-ahead, and @DSeaStar for reproducing it on Windows 11 / CPython 3.14 and checking the approach (strip the old hash before adding the new one, and rename only when the name would change).
Problem
Since 7.13.1 a parallel data file carries a hash of its data in its name (
.H<hash>h), which letscoverage combineskip duplicate files quickly.CoverageData.write()adds that hash only on the first write. If the process saves again after recording more data, the new data goes into the same file and the name keeps the first save's hash.Two processes that had the same data at their first save then end up with the same hash in their names, even though their later data differs.
combine_parallel_data()treats the second one as a duplicate, skips it, and deletes it, so the data it recorded after the first save is lost. Nothing warns; the combine message says "Combined 1 file, skipped 1".save()can be called more than once in a process (test_start_save_nostopsaves in the middle of a run), so a parallel run that saves at a checkpoint and then keeps measuring can hit this.Reproduction
Two separate processes, each
Coverage(data_suffix=True, source=["."]): start, importstartup.py(the same in both),save(), then importwork_a.pyin one andwork_b.pyin the other, stop,save(). Thencoverage combineandcoverage report:work_b.pyran but is reported as never run, and its data file is gone. Same result on 7.16.2 from PyPI and on main (99024b5). With 7.13.0, before the hash went into the name, both files are combined andwork_b.pyis 100%. Nothing is mocked or injected: real processes, real data files, the realcombinecommand.Change
In
CoverageData.write(), when a parallel data file already has a hash in its name and the hash of its data has changed since, rename it again with the new hash and update thehashrow inmeta(INSERT OR REPLACEinstead ofINSERT OR IGNORE). The first write is unchanged, a write with no new data does not rename, and files with the same data still get the same hash, so duplicates are still skipped. A forked child, which gets a new file name without a hash, is left as it was.Intended behaviour change: a parallel data file can be renamed on each
save()whose data changed, not only on the first.Testing
test_true_suffix_written_twice(test_data.py): two files share their first write, then differ; both must be combined. Fails on main, passes with the fix.test_combining_after_saving_twice(test_api.py): the same through theCoverageAPI, saving while still collecting. Fails on main, passes with the fix.test_true_suffix_written_twice_same_data(identical files written twice are still skipped as duplicates) andtest_true_suffix_written_again_without_new_data(no rename without new data).ruff format --check, pylint andmypy --strictare clean.