Skip to content

Accept two-letter byte units (KB/MB/GB/TB) in parse_size - #4468

Merged
hanouticelina merged 2 commits into
huggingface:mainfrom
Sreekant13:fix/parse-size-accept-two-letter-units
Jul 2, 2026
Merged

hanouticelina merged 2 commits into
huggingface:mainfrom
Sreekant13:fix/parse-size-accept-two-letter-units

Conversation

@Sreekant13

@Sreekant13 Sreekant13 commented Jul 1, 2026 •

Copy link
Copy Markdown
Contributor

The hf cache ls --filter help text and docs advertise size thresholds like size>1GB and size>10MB. The CLI's own --help example (and package_reference/cli.md) uses size>1GB, and manage-cache.md says these "work out of the box". But parse_size only accepted single-letter units, so the documented filters raised an error while the single-letter form worked:

from huggingface_hub.cli.cache import compile_cache_filter
compile_cache_filter("size>1GB", {})
ValueError: Unknown unit 'gb'. Must be one of ['k', 'm', 'g', 't', 'p'].
compile_cache_filter("size>30g", {}) # works
The sibling parse_size_to_int() in serialization/_base.py already supports TB/GB/MB/KB, so this also aligns the two parsers.

This adds the two-letter aliases (kb/mb/gb/tb/pb) to BYTE_UNITS and covers them in the parsing tests. 3KB was previously listed as an invalid case; it's now valid, alongside new 10MB/1GB/2TB cases. Duration parsing is untouched.

Happy to flip this the other way (fix the docs to use single-letter units) if you'd prefer to keep parse_size single-letter only.


Note

Low Risk
Small parsing-map and test updates with no auth, security, or data-path changes.

Overview
parse_size now accepts two-letter byte units (kb, mb, gb, tb, pb) in addition to the existing single-letter aliases, so documented hf cache ls --filter thresholds like size>1GB and size>10MB parse correctly instead of raising “Unknown unit”.

Tests add valid cases for 3KB, 10MB, 1GB, and 2TB, and drop 3KB from the invalid-size list. Duration parsing is unchanged.

Reviewed by Cursor Bugbot for commit 0914e52. Bugbot is set up for automated code reviews on this repo. Configure here.

Sreekant13 and others added 2 commits July 1, 2026 16:05
The `hf cache ls --filter` help and docs advertise size thresholds like
`size>1GB` and `size>10MB`, and the CLI's own --help example uses
`size>1GB`. But parse_size only accepted single-letter units, so those
documented filters raised `ValueError: Unknown unit 'gb'` while `size>30g`
worked. The sibling parse_size_to_int() already supports KB/MB/GB/TB.

Add the two-letter aliases so the advertised filters work, and cover them
in the parsing tests (3KB/10MB/1GB/2TB were previously untested or wrongly
marked invalid).

@hanouticelina hanouticelina left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for noticing and fixing! the other parser that we have in the already supports KB/MB/GB/TB, so yes let's align this one as well

@bot-ci-comment

bot-ci-comment Bot commented Jul 2, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@hanouticelina
hanouticelina merged commit a139277 into huggingface:main Jul 2, 2026
19 of 21 checks passed
@codecov

codecov Bot commented Jul 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.73%. Comparing base (1daa48b) to head (2f2b496).
⚠️ Report is 490 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4468      +/-   ##
==========================================
+ Coverage   75.00%   75.73%   +0.72%     
==========================================
  Files         145      175      +30     
  Lines       13978    22539    +8561     
==========================================
+ Hits        10484    17069    +6585     
- Misses       3494     5470    +1976     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Sreekant13

Copy link
Copy Markdown
Contributor Author

Thanks for the quick merge, @hanouticelina! Glad it lined up with parse_size_to_int. Happy to help align anything else if more of these turn up.

@huggingface-hub-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped as part of the v1.22.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants