Repository navigation
[Cache] Skip local-dir metadata when verifying checksums - #5257
Merged
hanouticelina merged 1 commit intoOct 27, 2026
Merged
hanouticelina merged 1 commit into
hanouticelina merged 1 commit into
Conversation
collect_local_files listed everything under the folder, including the .cache/huggingface/ metadata that downloads to a local_dir write, so hf cache verify --local-dir reported those files as extra and --fail-on-extra-files failed on every clean download. Filter with DEFAULT_IGNORE_PATTERNS, the same patterns uploads use.
hanouticelina
approved these changes
Oct 27, 2026
hanouticelina
left a comment
hanouticelina
left a comment
Collaborator
There was a problem hiding this comment.
Thanks a lot for fixing!
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5257 +/- ##
==========================================
+ Coverage 75.00% 77.67% +2.67%
==========================================
Files 145 178 +33
Lines 13978 25332 +11354
==========================================
+ Hits 10484 19677 +9193
- Misses 3494 5655 +2161 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Contributor
|
This PR has been shipped as part of the v2.4.0 release. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
hf cache verify --local-dirreports every file under.cache/huggingface/as an "extra" file, so an intact--local-dirdownload always warns, and--fail-on-extra-filesexits 1 even when every checksum matches. Fixes #5256.collect_local_files(used byverify_repo_checksums) listed everything under the folder viaroot.rglob("*"), including the metadata a--local-dirdownload writes:.gitignore,CACHEDIR.TAG,download/<file>.metadata,download/<file>.lock.verify_mapsplaced all of them inextra_paths.collect_local_fileswithDEFAULT_IGNORE_PATTERNS— the same patterns uploads already use to ignore.git/and.cache/huggingface/.test_collect_local_files_skips_local_dir_metadata; the existingtest_collect_local_files_lists_allstill passes.Before
Repro from the issue (
verify_mapson a cleanconfig.jsondownload) —res.extra_paths:After
res.mismatchesandres.missing_pathsstay empty in both cases.Note
Low Risk
Narrow change to local file enumeration for verification; reuses existing upload ignore patterns and adds a focused test.
Overview
collect_local_files(used by local-dir checksum verification) no longer treats Hugging Face download metadata as repo files. After building the full file map fromrglob, paths are filtered withDEFAULT_IGNORE_PATTERNSandfilter_repo_objects— the same rules uploads use to skip.git/and.cache/huggingface/(e.g..gitignore,CACHEDIR.TAG,download/*.metadata).That stops
hf cache verify --local-dirfrom fillingextra_pathsand failing--fail-on-extra-fileson otherwise valid--local-dirtrees. A regression test asserts metadata under.cache/huggingface/is omitted while other files under.cache/remain.Reviewed by Cursor Bugbot for commit 9891153. Bugbot is set up for automated code reviews on this repo. Configure here.