Repository navigation
Releases
[v2.4.0] Copy files into Buckets with `HfFileSystem`, outdated CLI extension hints
🪣 Copy files into Buckets with HfFileSystem
HfFileSystem raised NotImplementedError on any copy involving a Bucket. Copying into a Bucket now works from the same Bucket, from another Bucket, or from a model, dataset or Space repo. Files stored with Xet are copied on the Hub by hash, so nothing is downloaded. Other repo files are downloaded and then uploaded to the Bucket. fsspec moves a file by copying it and deleting the source, so mv and rename now work inside a Bucket too.
>>> from huggingface_hub import hffs
>>> hffs.cp_file("buckets/my-username/my-bucket/data/train.csv", "buckets/my-username/my-bucket/backup/train.csv")
>>> hffs.mv("buckets/my-username/my-bucket/data/tmp.csv", "buckets/my-username/my-bucket/data/final.csv")
>>> hffs.copy("buckets/my-username/my-bucket/data", "buckets/my-username/other-bucket/data", recursive=True)
>>> hffs.cp_file("datasets/my-username/my-dataset/train.parquet", "buckets/my-username/my-bucket/train.parquet")Copying from a Bucket to a repo is not supported yet.
- [HfFileSystem] Support same-bucket file copies by @ankitkumartiwari-web in #5080
🧩 See when a CLI extension is outdated
hf extensions list only showed when each extension was installed, so there was no way to know that an update was available. It now has a VERSION column (the commit the extension was installed from) and a LATEST column, plus a hint naming the outdated extensions. Running an outdated extension also prints a hint, at most once a day. To stay within GitHub's rate limits, hf looks up the latest commit of each extension at most once a day. Set HF_HUB_DISABLE_UPDATE_CHECK=1 to turn these checks off.
hf extensions update also repairs Python extensions that no longer start. After a Homebrew upgrade removed the Python an extension was installed with, running it failed with dyld: Library not loaded, and hf extensions update said it was already up to date. It now recreates the extension's virtual environment when its Python doesn't start, even if no newer commit exists.
$ hf extensions update speedtest
Creating virtualenv at: ~/.local/share/hf/extensions/hf-speedtest/venv
✓ Extension updated- [CLI] Show extension versions and hint when one is outdated by @hanouticelina in #5269
- [CLI] Make hf extensions update repair broken Python extensions by @hanouticelina in #5270
🔧 Other QoL improvements
- Add auth hint to HfFileSystem "repository not found" error by @davanstrien in #5258. A private repo accessed without a token gave the same
(repository not found)error as a typo in the repo name. The error now also tells you to check that you are authenticated.
🐛 Bug and typo fixes
- [Webhooks] Use "discussion" domain and default to all domains by @hanouticelina in #5263.
hf webhooks createfailed unless you passed--domain repo, and watching discussions was not possible because the accepted value wasdiscussionsinstead ofdiscussion.create_webhooknow watches both domains whendomainsis omitted. - Validate filename against path traversal in try_to_load_from_cache by @bodapatisaikrishna in #5249. Filenames with
../or absolute paths could resolve outside the cache folder. They are now rejected, as inhf_hub_download. - [HfFileSystem] Return file info from find(detail=True) on a file path by @ontdepart in #5246. It returned an empty dict, which broke datatrove readers pointing at a single
hf://file. - [Cache] Skip local-dir metadata when verifying checksums by @alphasafal in #5257.
hf cache verify --local-dirreported the.cache/huggingface/metadata as extra files, so--fail-on-extra-filesfailed on every clean download. - [CLI] Don't crash hf cache verify on Windows redirected stdout by @Sonica-B in #5255. When output was piped or redirected,
hf cache verifycrashed withUnicodeEncodeErrorinstead of listing the files that failed verification.
📖 Documentation
- [Docs] Fix docstring examples that fail when copy-pasted by @tkr109 in #5238
- [Docs] Await endpoint batch inference before pausing by @emrehangorgec in #5247
- Docs: build English only, other languages are machine-translated by @mishig25 in #5276. The docs in other languages are now machine-translated from the English docs, in 11 languages.
🏗️ Internal
- Post-release: bump version to 2.4.0.dev0 by @huggingface-hub-bot[bot] in #5240
- [Tests] Don't pin the avatar hash in OAuth test by @hanouticelina in #5241
- [CI] Drop urllib3<2.0 test pin broken by vcrpy 9 by @hanouticelina in #5262
- Bump the actions group with 7 updates by @dependabot[bot] in #5268
[v2.3.0] Bucket live-follow, faster `HfFileSystem` listings and more
🔄 Keep Bucket listings fresh with live_follow=True
HfFileSystem caches directory listings, so files added to a Bucket by another process did not show up until you called invalidate_cache(). With live_follow=True, HfFileSystem opens one background connection per listed Bucket to receive its file changes from the Hub. When a change arrives, it removes the cached listings it affects.
>>> from huggingface_hub import HfFileSystem
>>> hffs = HfFileSystem(live_follow=True)
>>> hffs.ls("buckets/my-username/my-bucket/data", detail=False) # starts following that bucket
>>> # ... another process uploads a file to the bucket ...
>>> hffs.ls("buckets/my-username/my-bucket/data", detail=False) # refreshed as changes arrive
['buckets/my-username/my-bucket/data/train.csv', 'buckets/my-username/my-bucket/data/new-file.csv']It also works through fsspec with fsspec.filesystem("hf", live_follow=True), and through the storage_options argument of libraries such as pandas and pyarrow. The option is off by default because it keeps a connection open for each followed Bucket. It only affects listings and metadata (ls, info, exists, glob, ...), because reading a file always fetches its content from the Hub. It is not available for model, dataset and Space repos: use refresh=True to see a new commit.
🚀 HfFileSystem lists large repos in parallel
Recursive listings such as find and glob used to be slow on large repos. The Hub returns a repo's file tree 1000 entries at a time, and each page can only be requested after the previous one has arrived. Listing datasets/allenai/objaverse (~800k files) took ~800 requests in a row, about 13 minutes.
HfFileSystem now splits recursive listings by sub-directory and lists them in parallel, with almost the same number of requests. A sub-tree that fits in a single page is never split. maxdepth is now applied during the listing, so the Hub only returns the levels you ask for.
datasets/allenai/objaverse |
before | after |
|---|---|---|
find(), 798,924 files |
801 requests, 776 s | 807 requests, 19.7 s |
find(maxdepth=2), 328 entries |
801 requests, 776 s | 6 requests, 1.5 s |
>>> from huggingface_hub import hffs
>>> files = hffs.glob("datasets/allenai/objaverse/**/*.glb", maxdepth=3)The new max_workers argument of HfFileSystem sets the number of concurrent requests. It defaults to 16. Set it to 1 to list sequentially like before.
📎 Attach images and videos to discussions and PRs
The new HfApi.upload_attachment(path) uploads an image, video or audio file and returns its URL, which you can embed in a discussion, PR description or comment. Before, this required calling an undocumented Hub endpoint by hand.
>>> from huggingface_hub import HfApi
>>> api = HfApi()
>>> url = api.upload_attachment("screenshot.png")
>>> api.comment_discussion("username/repo-name", 5, f"Here is the fix:\n\n")On the CLI, hf discussions create (including --pr), comment and edit take a repeatable --attach FILE[#alt text] option. If the body already references the file as , that reference is replaced with the uploaded URL. Other files are added at the end of the body, the same way the Hub web editor does it.
$ hf discussions create celinah/hfh-attach-test --title "Fix rendering" --pr \
--body "Before: " --attach ./before.png --attach "./after.png#After the fix"
num=1 url=https://huggingface.co/celinah/hfh-attach-test/discussions/1 ref=refs/pr/1
$ hf discussions info celinah/hfh-attach-test 1 --format json | jq -r '.events[0].content'
Before: 
- [Discussions] Add
upload_attachmentand--attachfor media files by @hanouticelina in #5161
🧩 Read every safetensors shard header in one request
get_safetensors_metadata reads the header of each shard of a sharded model separately. The new get_sharded_safetensors_metadata gets all of them from the Hub in a single request and returns the same SafetensorsRepoMetadata. It only reads the main revision. get_safetensors_metadata is unchanged.
>>> from huggingface_hub import get_sharded_safetensors_metadata
>>> metadata = get_sharded_safetensors_metadata("HuggingFaceH4/zephyr-7b-beta")
>>> len(metadata.files_metadata)
8- [Safetensors] Add
get_sharded_safetensors_metadatato get every shard header in one request by @mishig25 in #5159
🔒 Access requests on gated repos that hide requester identity
Owners of gated repos can now choose not to share the username and/or email of the people requesting access. On those repos, list_pending_access_requests (and the accepted and rejected variants) failed with a KeyError. Each AccessRequest now has a request_id, and accept_access_request, reject_access_request and cancel_access_request accept request_id= instead of user.
>>> from huggingface_hub import list_pending_access_requests, accept_access_request
>>> for request in list_pending_access_requests("my-org/gated-model"):
... accept_access_request("my-org/gated-model", request_id=request.request_id)Note
AccessRequest.username and AccessRequest.fullname are now typed str | None. They are only None on repos that hide usernames, where listing requests used to fail anyway, but type checkers may flag code that assumes a str.
🛡️ create_commit no longer deletes files you re-add in the same commit
create_commit skips adds and copies whose content is already on the Hub. When the same commit also deleted that path or its parent folder, only the delete was sent, and the file was removed from the repo without any error. This affected deleting a folder and pushing the same files back in one commit, and had been happening since v0.24. Those adds are now kept.
api.create_commit(repo_id, operations=[
CommitOperationDelete("x/", is_folder=True),
CommitOperationAdd("x/a.txt", b"keep me\n"), # unchanged on the Hub
], commit_message="Reset x/")
# before: x/a.txt is deleted
# after: x/a.txt is kept- [HfApi] Keep unchanged files re-added after a delete in create_commit by @hanouticelina in #5196
💼 hf jobs run keeps streaming logs and shows queue status
hf jobs run stopped printing logs when the log stream ended before the Job did, and crashed after a few dropped connections. It now reconnects until the Job reaches a final stage. Ctrl+C used to stop the logs while the CLI kept waiting silently for the Job to end. It now exits right away and tells you that the Job is still running and how to follow or cancel it.
While a Job waits for hardware, hf jobs run, hf jobs uv run and hf jobs rerun printed nothing, so a queued Job looked like a stuck CLI. They now show the Hub's status message and how long the Job has been waiting, in interactive terminals only:
- [Jobs] Fix hf jobs run losing logs and ignoring Ctrl+C by @hanouticelina in #5162
- [CLI] Show the Job's scheduling status in hf jobs run by @hanouticelina in #5163
🐍 Python 3.15 support
huggingface_hub now officially supports Python 3.15 and is tested on it in CI. No code changes were needed. Python 3.10 is still the minimum version.
- [CI] Add CI + official support for Python 3.15 by @hanouticelina in #5155
🤖 Inference
- [Inference] Don't keep non-streaming responses alive in InferenceClient by @hanouticelina in #5197. A reused
InferenceClientkept every non-streaming response in memory untilclose(). With 30 calls and a 1 MB prompt, that was 30 MB. - [Inference] Run provider get_response in a thread in AsyncInferenceClient by @hanouticelina in #5200. Queue-based providers (fal-ai, wavespeed, together video, zai-org) wait for results with blocking calls, which froze the event loop for the whole job.
- [Inference] Raise InferenceTimeoutError on httpx timeouts by @hanouticelina in #5150.
timeoutnow raisesInferenceTimeoutErroras documented, instead of the raw httpx timeout error.
🖥️ CLI
- [Cache] Show unreferenced shared blobs in
hf cache lsby @hanouticelina in #5148. Files left behind when an olderhuggingface_hubversion deletes a repo now show up as a hint pointing tohf cache prune. - [CLI] Fix hf skills add when .claude/skills links to .agents/skills by @hanouticelina in #5204
🔧 Other QoL improvements
- [Jobs] Export list_scheduled_jobs from the top-level package by @guptaparin in #5140
- [Bot] Update hardware flavor enums and docs by @huggingface-hub-bot[bot] in #5175
🐛 Bug and typo fixes
- fix(kernels): allow uploads and commits to kernel repos by @tengomucho in #5233
- [HfApi] Respect custom endpoint in webhooks, access requests and user APIs by @hanouticelina in #5149. With
HfApi(endpoint=...), 20 methods (webhooks, access requests, user and org info, followers, members) were sent to huggingface.co with the instance's token. Setups usingHF_ENDPOINTwere not affect...
[v2.2.0] `hf jobs stats` snapshots, download and `HfFileSystem` fixes
A small release this week, mostly bug fixes and quality-of-life improvements. hf jobs stats now returns right away instead of streaming forever, concurrent downloads reliably use hf_xet, and HfFileSystem caching is more predictable. Two behavior changes are listed under Breaking changes.
📊 hf jobs stats prints a snapshot by default
hf jobs stats used to stream metrics until the job finished, so scripts and agents calling it on a running job hung until their own timeout. It now prints the latest sample of each job once and exits, like hf jobs logs and hf spaces logs. Use -f/--follow for the previous live view. Snapshot output also supports --format json|agent|quiet, which outputs the raw API metrics.
On the Python side, HfApi.fetch_job_metrics gets the same follow parameter as fetch_job_logs. It defaults to False and returns only the current sample. Pass follow=True to stream until the job completes.
>>> hf jobs stats 6ac3bd9afbc85ba6823a9ad4
JOB ID CPU % NUM CPU MEM % MEM USAGE NET I/O GPU UTIL % GPU MEM % GPU MEM USAGE
--------------------- ----- ------- ----- -------------- --------------- ---------- --------- -------------
6ac3bd9afbc85ba682... 0% 2.0 0.01% 1.0MB / 16.0GB 0.0bps / 0.0bps N/A N/A N/A
Hint: Use `hf jobs stats -f celinah/6ac3bd9afbc85ba6823a9ad4` to follow live stats.
# Stream live stats until the job completes
>>> hf jobs stats -f 6ac3bd9afbc85ba6823a9ad4>>> from huggingface_hub import fetch_job_metrics
>>> for metrics in fetch_job_metrics(job_id=job.id, follow=True):
... print(metrics)Other Jobs fixes in this release: hf jobs logs and hf spaces logs without -f no longer hang while the job keeps printing, hf jobs stats no longer fails when one of the requested jobs has finished, and hf jobs stats --json prints [] when no job is running.
- [CLI] Snapshot by default in hf jobs stats, add -f/--follow by @hanouticelina in #5090
- [CLI] Fix empty hf jobs stats --json output with no running jobs by @hanouticelina in #5094
- [Jobs] Fix no-follow logs hanging while the job keeps printing by @hanouticelina in #5092
- [Jobs] Ignore 406 from metrics endpoint for finished jobs by @hanouticelina in #5091
📚 Documentation: Jobs guide
⚡ Concurrent downloads always use hf_xet
In a fresh process, the first concurrent hf_hub_download calls could read a placeholder version for hf_xet while another thread was still looking it up. Those calls treated hf_xet as missing and downloaded through the plain HTTP path. This mostly affected parallel downloads such as snapshot_download, where some of the files skipped Xet. The version lookup is now cached only once it is resolved, so every download sees the real value.
- Fix race in
_get_versionthat makes concurrent downloads skip hf_xet by @BramVanroy in #5102
📂 HfFileSystem cache fixes
Three fixes to how HfFileSystem caches listings and missing paths:
ls()on a newly created empty bucket returns an empty list instead of raisingFileNotFoundError.- Directory listings are now replaced in the cache instead of appended to. Before,
ls()afterfind()orls(refresh=True)could return duplicate entries,refresh=Truekept deleted files, and listing a single bucket file cached it as the whole folder's listing. exists(path, refresh=True)andinvalidate_cache(path)now clear cached "not found" results. Before, a repo, branch or bucket created in the same process stayed missing until a fullinvalidate_cache().
>>> fs.exists(repo_id)
False
>>> api.create_repo(repo_id)
>>> fs.exists(repo_id, refresh=True) # was False before this release
True- [HfFileSystem] fix
ls()on new empty bucket by @lhoestq in #5097 - [HfFileSystem] Replace cached listings instead of appending to them by @ChihebBENCHEIKH1 in #5099
- [HfFileSystem] Make refresh=True clear cached not-found results by @hanouticelina in #5106
💔 Breaking changes
-
hf jobs statsandHfApi.fetch_job_metricsreturn a single snapshot by default instead of streaming until the job completes. Use-f/--followorfollow=Trueto keep the previous behavior. See the section above (#5090). -
load_torch_modelnow raisesFileNotFoundErrorinstead ofValueErrorwhen the checkpoint path does not exist, andmodel_index_to_eval_resultsraisesValueErroron an empty list (#5063). -
[Bugfix] Fix wrong exception type and unbound variable guard by @vector15-05 in #5063
🖥️ CLI
- [CLI] Don't glob-expand arguments on Windows by @hanouticelina in #5117. On Windows, quoted patterns such as
--exclude ".venv/**"or--include "*.json"were expanded into local file names before reaching the command.hf download <repo> --include "*.json"in a folder containing a.jsonfile silently downloaded nothing. - [CLI] Read --env-file / --secrets-file as UTF-8 and strip the BOM by @hanouticelina in #5066
- [CLI] Upload existing paths with glob characters as-is in hf upload by @Ganesh2325 in #5059
🔧 Other QoL improvements
- [Download] Warn when allow/ignore patterns match no files in snapshot_download by @Ad1th in #5067
- [Cache] Speed up delete_revisions with a set lookup by @hanouticelina in #5118. Planning
hf cache prune/hf cache rmno longer scales quadratically with the number of files per revision. - [Setup] Install hf-xet on Windows ARM64 by @angt in #5074
🐛 Bug and typo fixes
- [Download] Fix offline snapshot_download with a pinned revision by @ChihebBENCHEIKH1 in #5113
- [Download] Consider port when following /resolve redirects by @assafvayner in #4942
- [CommitScheduler] Fix race that skips the first scheduled push by @hanouticelina in #5086
- [Repocard] Respect configured endpoint when validating cards by @Wauplin in #5069
- Fix argument name in raise_text_generation_error docstring by @Sreekant13 in #5108
📖 Documentation
- [Docs] Fix HTTP migration and async session references by @Yuilona in #5079
- [Docs] Fix from_pretrained integration example by @Ayushdevo in #5105
🏗️ Internal
- Post-release: bump version to 2.2.0.dev0 by @huggingface-hub-bot[bot] in #5054
- [CI] Split tests into api / transfer / inference suites by @Wauplin in #5042
- [Tests] Always clean up staging repos and buckets created by tests by @Wauplin in #5057
- [CI] Reduce flaky tests (jedi cache, read-after-write, fewer reruns) by @Wauplin in #5058
- Bump doc-builder workflow pin by @paulinebm in #5087
- Bump doc-builder main docs workflow pin by @paulinebm in #5093
- [Tests] Wait for bucket listings after writes by @hanouticelina in #5095
v2.1.1
[CLI] Rename 'hf spaces zero-gpu-quota' to 'hf spaces zero-gpu quota'- #5055
Full Changelog: v2.1.0...v2.1.1
[v2.1.0] Better Jobs tooling, ZeroGPU quota tracking, and revamped Model Catalog
📊 Jobs: retries, rescheduling, and port exposure
Hugging Face Jobs get a batch of quality-of-life improvements this release. Jobs can now retry automatically on failure with the new attempts parameter, and you can start a fresh Job from an existing Job's saved spec (including secrets and hardware settings) with rerun_job / hf jobs rerun. Scheduled Jobs can be rescheduled — change the cron expression or preset without recreating the Job — via update_scheduled_job_schedule / hf jobs scheduled reschedule. Finally, you can now expose ports on Jobs: use expose for token-protected access and expose_public for unauthenticated access, both at creation time and on a running Job with update_job_expose / hf jobs expose.
# Retry a job up to 2 times on failure, and rerun it later from its saved spec
>>> hf jobs run --attempts 3 --detach python:3.12 python train.py
>>> hf jobs rerun <job_id>
# Reschedule a scheduled job to run every day at 9:00
>>> hf jobs scheduled reschedule <scheduled_job_id> "0 9 * * 1"
# Expose ports on a running Job (8000 token-protected, 9000 public)
>>> hf jobs expose <job_id> 8000 --public 9000- [Jobs] Add rerun and retry attempts by @Wauplin in #5025
- [Jobs] Allow rescheduling scheduled jobs by @Wauplin in #5026
- [Jobs] Manage public and private exposed ports by @Wauplin in #5027
📚 Documentation: Jobs guide
🤗 Track your ZeroGPU quota
The Hub now exposes a ZeroGPU quota endpoint, and huggingface_hub wraps it with a new HfApi.get_zero_gpu_quota() method returning a ZeroGpuQuota dataclass (values in GPU-seconds). This is particularly useful for apps, MCP servers and agents built on top of ZeroGPU Spaces: check how much quota is left and when it resets, and warn your users before they hit the daily limit. The same information is available from the CLI with hf spaces zero-gpu-quota, which also prints a hint to purchase credits when the quota is running low.
>>> from huggingface_hub import get_zero_gpu_quota
>>> quota = get_zero_gpu_quota()
ZeroGpuQuota(base=2400, remaining=1810, resets_at=datetime.datetime(2026, 9, 30, 9, 12, 3, tzinfo=datetime.timezone.utc), overquota_used=0)>>> hf spaces zero-gpu quota
✓ ZeroGPU quota (in GPU-seconds)
remaining: 1810
base: 2400
resets_at: 2026-09-30T18:12:03+00:00
overquota_used: 0- [Spaces] Add get_zero_gpu_quota and hf spaces zero-gpu-quota by @Wauplin in #5040
- [CLI] Rename 'hf spaces zero-gpu-quota' to 'hf spaces zero-gpu quota'- #5055 [Shipped as part of v2.1.1]
📚 Documentation: Manage your Space guide
⚡ HfFileSystem.get_file downloads Xet files with hf_xet
HfFileSystem.get_file now downloads Xet-backed files through hf_xet (the same code path as hf_hub_download) when writing to a local path, letting hf_xet write directly to disk instead of going through Python. It reuses the Xet hash already returned by info() (part of the tree listing), so no extra HTTP call is needed. This mostly benefits non-streaming datasets loads from hf://buckets/... paths: benchmarks on HF Jobs show get_file dropping from ~120s to ~7s on a 2 GB parquet file, and load_dataset("hf://buckets/...") from ~130s to ~17s. The classic HTTP path is still used for non-Xet files, when hf_xet is not installed, or when writing to a file-like object. As a side effect, a missing remote file no longer leaves an empty local file behind.
- [HfFileSystem] Download Xet files with hf_xet in get_file by @hanouticelina in #4990
🧠 Inference Endpoints: Model Catalog migrates to the new v1 API
The catalog methods now use the new /api/v1/catalog API. list_inference_catalog returns InferenceCatalogModel dataclasses with their tested deployment recipes (hardware + engine combinations), and takes server-side filters (accelerator, engine, license, task, search, limit). create_inference_endpoint_from_catalog can deploy an exact recipe with the new recipe_id and gguf_file arguments. The CLI follows suit: hf endpoints catalog ls prints one row per recipe and supports the filters, while hf endpoints catalog deploy gains --recipe and --gguf-file.
>>> hf endpoints catalog ls --engine vllm --task text-generation --search llama --limit 10
REPO_ID TASK LICENSE ACCELERATOR ENGINE GGUF_FILE RECIPE_ID
----------------------------------- ------------------ ---------- ----------- -------- ----------------------------- -------------------------
meta-llama/Llama-3.1-8B-Instruct text-generation Llama 3.1 gpu vllm sizzling-biryani-g4xsi1ac
bartowski/QwQ-32B-Preview-GGUF text-generation Apache 2.0 gpu llamacpp QwQ-32B-Preview-Q8_0.gguf baked-orange-m863gx7d📚 Documentation: Inference Endpoints guide
🔒 Sandboxes require scoped credentials
Sandbox security is tightened in this release. Pooled sandboxes with missing, empty or malformed capability tokens are now refused instead of silently substituting the host-management credential, and the client is pinned to sbx-server 0.7.0, which removes host-token acceptance on per-sandbox routes entirely. Older pool hosts that do not provide scoped tokens must be upgraded/recycled: new clients cannot reach legacy hosts and legacy clients cannot reach new sandboxes (documented, accepted breaking change). The sandbox concepts and guides have been updated to describe the strict behavior, including how to safely use proxy_headers with external HTTP clients (disable redirects!).
- [Sandbox] Require scoped credentials for pooled sandboxes by @Wauplin in #4782
- [Sandbox] Bump sbx-server to 0.7.0 by @Wauplin in #5028
💔 Breaking Change
Note
The Inference Endpoints catalog migration (#4928) and the Sandbox credential changes (#4782, #5028) are breaking changes detailed in their highlight sections above.
🖥️ CLI
- [CLI] Escape control characters in listings and table output by @Wauplin in #5029
- [CLI] Suggest
hf auth loginwhenhf auth whoamifails by @moon-bot-app[bot] in #5033 - [CLI] Align
hf auth whoaminot-logged-in message withhf auth tokenby @Wauplin in #5034 - [CLI] Raise if a bare --secrets NAME is not set locally in hf jobs by @Wauplin in #5044
- [CLI] Hint to use --global after local 'hf skills add/update' by @Wauplin in #5024
- [CLI] nit: update
hf uploadhelp text by @davanstrien in #5046
🔧 Other QoL Improvements
- [HfApi] Support resource_group_id in duplicate_repo by @Wauplin in #5038
- [Repos] Warn when duplicated repo files are still being copied by @Wauplin in #5041
- Support secrets on Job-triggered webhooks by @moon-bot-app[bot] in #5009
- [HfApi] Add
visibilitytocreate_bucketby @hanouticelina in #5053
🐛 Bug and typo fixes
- [Buckets] Fix copy_files treating lexical siblings as an existing destination directory by @AnishPatel526 in #5001
- [HTTP] Fix repo_id in errors for query strings and download URLs by @Wauplin in #5008
- Fix scalar tags splitting into single-character list in card front matter by @zeenat28-ui in #5017
- [datasets sql] Run the DuckDB CLI subprocess with UTF-8 encoding by @825pranav in #5020
- [Auth] Store token names as UTF-8 instead of locale encoding by @Evolian-o in #5021
- [Dataclasses] Accept bare typing.List/Dict/Set in @strict validation by @anishmehta24 in #5032
- [Utils] Retry the first pagination page as well by @Wauplin in #5035
📖 Documentation
- [docs] xet caching and concurrency updates by @cbensimon in #4883
- [Spaces] Update docs for cpu-basic gating and dev mode by @Wauplin in #5037
- [docs] Fix hf.cos typo in repository guide link by @pratikgx in #5016
- [Docs] Fix links in i18n issue template by @pratikgx in #5051
- [Docs] Fix duplicated words in docstring and comments by @Ad1th in #5052
🏗️ Internal
[v2.0.0] The httpx2 release
See MIGRATION GUIDE in docs.
💔 Breaking Change: HTTP stack moves to httpx2
See migration plan in #4802 .
huggingface_hub v2.0 replaces its HTTP dependency with httpx2: clients and transport exceptions now come from httpx2/httpcore2 instead of httpx/httpcore. Custom client factories (via set_client_factory / set_async_client_factory) and any except clauses catching transport errors must use httpx2 types. For code that must support both v1.x (starting with v1.30.0) and v2.x, import the HTTP module from huggingface_hub.utils:
from huggingface_hub.utils import get_session, httpx
try:
response = get_session().get("https://huggingface.co/api/models/gpt2")
response.raise_for_status()
except httpx.HTTPError:
...Note that httpx2 uses the operating system's certificate trust store by default (custom CA bundles via SSL_CERT_FILE / SSL_CERT_DIR still work), logging configuration should target the httpx2 and httpcore2 loggers, and the oauth extra now requires authlib>=1.8.0.
💔 Breaking Change: deprecated APIs are removed
This release also removes all APIs deprecated in 1.x:
upload_large_folder/HfApi.upload_large_folder→ useupload_folderduplicate_space→ useduplicate_reporequest_space_storage/delete_space_storage→ use the volume APIs (set_space_volumes,delete_space_volumes)repo_type_and_id_from_hf_id→ useparse_hf_uriInferenceEndpointType.PROTECTED→ useAUTHENTICATED(create_inference_endpoint(type="protected")now raisesValueError)list_models(model_name=...)→ uselist_models(search=...)create_repo/duplicate_repo(space_storage=...)→ usespace_volumestext_generation(stop_sequences=...)→ usestop- Boolean
InferenceClient(token=...)→ pass a token string orNone
On the CLI side, the following entry points and options are gone: huggingface-cli (use hf), hf repo and hf repo-files delete (use hf repos and hf repos delete-files), hf upload-large-folder (use hf upload), hf repos create/duplicate --storage (use --volume), hf jobs ps and hf jobs scheduled ps -f/--filter (use --status/--label), and hf skills add/update --claude (Claude Code installation is now automatic).
A migration guide covering all of this is available at https://github.com/huggingface/huggingface_hub/blob/main/docs/source/en/concepts/migration_v2.md.
🏗️ Internal
[v1.33.0] Better UX on large uploads and simpler skills installs
📤 See progress while large Xet uploads validate
Very large Xet uploads used to sit silently between "Uploading" and "Committing" while shards were being finalized, which could take a long time with no feedback. The upload live display now shows a dedicated Validating bar — reported as a percentage instead of opaque internal validation-entry counts — both in the upload_folder display and in XetUploadProgressReporter (used by upload_file, create_commit and bucket uploads). This requires hf-xet>=1.6.0.
🧩 Skills are now installed for every agent by default
hf skills add no longer requires a special flag for Claude Code. The skill is now always installed to .agents/skills (or ~/.agents/skills with -g) and symlinked into .claude/skills (honoring CLAUDE_CONFIG_DIR when set), so a single command covers Claude Code, Codex, Cursor, OpenCode, Pi and any other agent that loads skills from .agents/skills. The --claude flag is deprecated: it still works as a no-op but prints a warning, and --dest keeps installing only to the directory you provide. hf skills update refreshes both roots the same way. Along the way, CLAUDE_CONFIG_DIR is now properly honored everywhere (it was previously ignored, which left skills invisible to relocated Claude Code installs).
# works with Claude Code, Codex, Cursor, OpenCode, Pi and any agent that loads skills from `.agents/skills`
hf skills add- [CLI] Always install skills for Claude Code, deprecate --claude by @hanouticelina in #4967
- [CLI] Honor
CLAUDE_CONFIG_DIRinhf skills add -g --claudeby @hanouticelina in #4957
📚 Documentation: CLI guide
💔 Breaking Change
- Send PrivateLink config under privateService in create_inference_endpoint by @co42 in #4966
- AWS PrivateLink settings are now sent as
privateService.accountIdandprivateService.region. Use the newprivate_link_account_idandprivate_link_regionparameters (the region is required when the account ID is set, and is independent of the compute region). The legacyaccount_idkeyword still exists but is ignored and emits aFutureWarning; it is no longer included as a top-level payload field.
- AWS PrivateLink settings are now sent as
🖥️ CLI
- [CLI] Honor --format / --json / -q on jobs run commands by @Wauplin in #4936 — docs
- These flags previously leaked into the container command on
hf jobs run,hf jobs uv runand theirscheduledvariants (--formatbecame the script path,-qbecame the image name). They are now consumed wherever they appear; use--when your script needs them.
- These flags previously leaked into the container command on
- [CLI] Allow
bucketas a watched item type inhf webhooks create --watchby @davanstrien in #4958 — docs
📖 Documentation
- [Docs] Sync provider task table with the PROVIDERS registry by @rishabhsai in #4930 — docs
- Remove dots from allowed characters in job labels (docstring update) by @evalstate in #4962
🐛 Bug and typo fixes
- [Download] Publish symlink support only after probing by @Wauplin in #4923
- [Core] Skip invalid eval results entries instead of raising by @davanstrien in #4924
- [Cache] Use posix paths for
scan_cache_dirref names on Windows by @gokay-ai in #4925 - [Download] Measure http_get progress from the initial file position by @Wauplin in #4937
- Fix
format_timesinceshowing "60 seconds ago" / "60 minutes ago" / "24 hours ago" instead of rolling over by @Sreekant13 in #4943 - [HfFileSystem] Make lpath optional in get_file (fsspec>=2026.9.0) by @assafvayner in #4951
- [Buckets] Raise
BucketBatchErrorwhen a batch reports failed operations by @hanouticelina in #4954 — docs - Align the
hfpackage'spython_requireswith huggingface_hub by @sharziki in #4959 - [MCP] Follow nextCursor when listing tools in MCPClient by @Kayvan-Zahiri in #4960
- [Download] Fix Windows MAX_PATH error on tmp .incomplete file by @Wauplin in #4978
- [HfFileSystem] Fix get_file with a bare local filename by @davanstrien in #4981
- Apply the repo_id length limit to the namespace as well by @anishmehta24 in #4988
🏗️ Internal
- Post-release: bump version to 1.33.0.dev0 by @huggingface-hub-bot[bot] in #4926
- [Serialization] Fix device type when map_location is None by @Wauplin in #4938
- [Internal] Soft deprecate
re.matchby @Wauplin in #4939 - Fix Windows conftest erroring test subsets on lazy-loaded file_download submodule by @dltsum in #4945
- Scope GITHUB_TOKEN permissions per job by @hf-security-analysis[bot] in #4982
[v1.32.0] Shared blob store, sandbox security hardening and faster imports
📂 [Cache] Shared blob store: deduplicate Xet files across repos
The cache now deduplicates Xet files across repos. A Xet file downloaded through hf_xet is stored once at <CACHE_DIR>/blobs/<prefix>/<xet_hash> and every repo that needs it gets a relative symlink instead of a download: no bytes are transferred and no extra space is used, even across different repos — or after the repo that first downloaded the file was deleted. The per-repo snapshot layout is unchanged, older clients keep reading and downloading normally, and any failure to share silently falls back to regular repo-local storage. Set HF_HUB_DISABLE_SHARED_BLOBS=1 to opt out entirely. Shared files carry a <xet_hash>.refs manifest listing the repo blobs referencing them, which hf cache rm consults on deletion and hf cache prune sweeps to reclaim payloads that no cached repo uses anymore.
📚 Documentation: Manage your cache
- [Cache] Add cross-repo shared blob store by @hanouticelina in #4498
💻 [Jobs] Ship config inside UV scripts
A UV script that only runs correctly on a specific runtime can now carry that runtime with it. An optional [tool.hf-jobs] table in the script's PEP 723 header accepts image, flavor, python, timeout, name, namespace, env, secrets, labels, volumes, network_group and network_aliases, and hf jobs uv run reads it at submit time. CLI flags always win, and env/secrets/labels/volumes merge entry by entry instead of being replaced, so -e/-v add to what the script declares. Typos and unknown keys are rejected with the list of valid options, secrets are passed by name only (values come from your environment), and every run prints a config summary with script-sourced values marked and secrets redacted. Note that the table is read by the CLI only: run_uv_job() and create_scheduled_uv_job() ignore it.
# /// script
# requires-python = ">=3.11"
# dependencies = ["vllm", "datasets"]
#
# [tool.hf-jobs]
# image = "vllm/vllm-openai:unlimited-ocr"
# flavor = "l4x1"
# python = "/usr/bin/python3"
# secrets = ["HF_TOKEN"]
# ///📚 Documentation: Run and manage Jobs
🛡️ [Sandbox] security hardening
An internal security review of the Sandbox API resulted in a 9-PR hardening series. Pooled sandboxes now use their own per-sandbox capability token instead of the host-wide credential, pool hosts are fully validated (initiator, namespace, image, flavor, command, URL) before any credential is sent to them, the sbx-server binary is pinned by digest and verified before being run as PID 1, and the local pool cache is bound to the endpoint, credential and namespace that wrote it. Secret values no longer end up in argv when using hf sandbox exec --secrets, background processes are addressed by their server-assigned id (so kill() actually stops them and reports honestly), transfers and command output are bounded to avoid unbounded memory usage in the client, and pool ownership is decided per host so a with SandboxPool(...) block never again tears down a colleague's discovered host. The security documentation was also rewritten to state precisely what the sandbox contract is — and what it is not — including a new "Known limitations" section.
📚 Documentation: Sandboxes
- [sandbox audit] Make the sandbox security contract match the implementation by @Wauplin in #4831
- [sandbox audit] Use each pooled sandbox's own capability token by @Wauplin in #4832
- [sandbox audit] Validate a pool host before sending it a credential by @Wauplin in #4834
- [sandbox audit] Bind the sandbox pool cache to the endpoint, credential and namespace that wrote it by @Wauplin in #4838
- [sandbox audit] Bound what a transfer or a command's output costs the client by @Wauplin in #4839
- [sandbox audit] Decide host teardown per host, and report it honestly by @Wauplin in #4840
- [sandbox audit] Address background processes by their server-assigned id by @Wauplin in #4836
- [sandbox audit] Pin the sandbox server binary by digest and verify it before running it by @Wauplin in #4837
- [sandbox audit] Keep secret values out of argv in the CLI and fix the env/secrets docs by @Wauplin in #4835
🔒 Security hardening for serialization and validation
Three changes make the library safer against malicious or malformed inputs, with a stricter behavior worth noting. Torch checkpoint deserialization was hardened across 11 reported vulnerabilities: load_state_dict_from_file now defaults to safe=True (always using the safetensors loader), the pickle path defaults to weights_only=True, and unsafe combinations raise explicit errors instead of silently falling through. Path validation now rejects .. segments anywhere in path_in_repo (previously only a leading one was caught), so uploads like "a/../../etc/passwd" are refused. Finally, repo_id validation is restricted to ASCII word characters as documented, so non-ASCII ids like café are rejected client-side instead of failing later on the Hub.
- [Serialization] Harden torch checkpoint deserialization by @Wauplin in #4877
- [Core] Reject path traversal via embedded ".." segments in path_in_repo by @bodapatisaikrishna in #4884
- [Core] Restrict repo_id validation regex to ASCII word characters by @bodapatisaikrishna in #4892
🧰 Smarter cache management
hf cache rm accepts hf:// file URIs, so you can remove a single cached file — typically one GGUF quantization — without deleting the whole repo. The file is removed from every cached revision, and its blob is deleted only if no other cached file still references it. hf cache commands also support kernel repos: scan_cache_dir() used to treat kernels--* folders as corrupted and skip them, but kernel repos now show up in hf cache ls, rm and prune, and --repo-type kernel works for hf download and hf cache verify.
hf cache rm hf://models/unsloth/gemma-3-27b-it-GGUF/gemma-3-27b-it-Q4_K_M.gguf --dry-run
hf download kernels-community/activation README.md --repo-type kernel📚 Documentation: Manage your cache
- [CLI] Support file URIs in hf cache rm by @hanouticelina in #4847
- [Cache] Support kernel repos in
hf cachecommands by @hanouticelina in #4905
🚀 Faster imports
huggingface_hub now imports much faster: huggingface_hub.utils is lazy like the root package, heavy HfApi implementation imports and optional Torch/TensorBoard/NumPy/FastAPI/Starlette dependencies are deferred until used, and CLI commands are registered lazily while hf --help still renders the same complete output. Benchmark medians on Python 3.10: from huggingface_hub import * drops from 2.36s to 0.39s (626.8 → 49.6 MiB peak RSS), hf version from 0.37s to 0.17s, and hf env from 0.40s to 0.24s.
🖥️ CLI
- [CLI] Make --format human override agent auto-detection by @hanouticelina in #4878
- [CLI] Add
hf --skillsas an alias forhf skills previewby @Wauplin in #4906
🤖 Inference
- Add
--container-registry-{username,password}by @alvarobartt in #4854 — docs
📊 Jobs
- [Jobs] Scope network groups to the namespace and resource group by @co42 in #4855
- [Jobs] Expose the network group on JobInfo by @co42 in #4862
🔧 Other QoL Improvements
- [Download] Send X-HF-Download-Counter header on download calls by @Wauplin in #4613
- [Buckets] Send mtime when copying files by @Wauplin in #4920
🐛 Bug and typo fixes
- fix(serialization): save_torch_state_dict cleanup regex silently matches/deletes unrelated files by @g0rdonL in #4857
- [Sandbox] Preserve terminal command results by @evalstate in #4851
- [Http] Fix race in
get_session()creating multiple clients by @hanouticelina in #4868 - [RepoCard] Fix AttributeError when model-index has source: null by @Saniyagupte in #4870
- [Spaces] Fix AttributeError in SpaceRuntime when hardware is null by @Saniyagupte in #4873
- [Buckets] Fix file metadata size on redirects by @davanstrien in #4874
- [CLI] Fix card data serialization by @davanstrien in #4882
- Fix doubled colon in snapshot download transfer bar description by @sandeepb2013 in #4890
- [Download] Fix silently disabled tree cache on long Windows paths by @askalf in #4896
- [Safetensors] Recognize proposed U3 metadata by @7dsolv in #4898
- [Serialization] Sync
_get_dtype_sizewith safetensors by @Wauplin in #4902
📖 Documentation
- [Community] Fix docstring for DiscussionComment.last_edited_by by @Saniyagupte in #4864
- [Docs] Fix hf discussions info options that do not exist by @rishabhsai in #4886
🏗️ Internal
- [CI] Make CI less flaky and stop 6h hung jobs by @Wauplin in #4830
- Bump the actions group with 4 updates by @dependabot[bot] in #4848
- Post-release: bump version to 1.32.0.dev0 by @huggingface-hub-bot[bot] in #4856
- Key the doc build concurrency group on the pull request number by @albertvillanova in #4876
[v1.31.0] Custom labels for Sandboxes, More resilient downloads and more
🏷️ Custom labels for Sandboxes
Dedicated sandboxes now accept custom labels, attached to the underlying Job. This is useful for cost attribution, bookkeeping, or finding and reconnecting to the sandboxes created by a given controller run. Labels use the same -l / --label KEY=VALUE syntax as hf jobs run. They are merged with the labels the SDK uses internally, and invalid or reserved labels are rejected before a billable Job is started. Pool-based sandboxes are unchanged: custom labels are only accepted for dedicated sandboxes.
>>> from huggingface_hub import Sandbox
>>> sandbox = Sandbox.create(image="python:3.12", labels={"controller-run": "run-42"})hf sandbox create --label controller-run=run-42 --label team=data-infra📥 More resilient downloads
A batch of fixes makes downloads more robust to unusual server responses, network hiccups and concurrent usage:
-
A timeout while waiting for the response headers of a streamed download is now retried and resumed like a body read failure, instead of escaping
http_get()while retries remain. -
Regular HTTP downloads no longer fail when the HEAD response has no
Content-Length. The file size is validated against the GET response when available. -
snapshot_downloadnow writes therefs/cache file atomically, reusing the logic already used byhf_hub_download. This fixes a long-standing race when many concurrentsnapshot_downloadcalls target the same repo (seen in vLLM /llm-compressor). -
hf download --dry-run --local-dir ...no longer copies files from the Hub cache into the destination. On large files and slow disks this looked like a hang and could leave an incomplete file behind. -
[Download] Share retry handling for stream entry and body failures by @Wauplin in #4826
-
[Download] Tolerate missing HEAD Content-Length by @Wauplin in #4805
-
[Download] Write cache ref file atomically in
snapshot_downloadby @Wauplin in #4829 -
[Download] Prevent cached file copies during local-dir dry runs by @wakamex in #4817
🔌 httpx re-exported for library integrators
huggingface_hub now re-exports the HTTP library it uses as huggingface_hub.utils.httpx. Libraries built on top of huggingface_hub that need httpx types or exceptions (typically to catch errors) should import it from there rather than importing httpx directly. v1.x is built on httpx, and v2.x will move to its successor httpx2, so importing through huggingface_hub.utils keeps your code compatible with both. This is only for types and exceptions: to make requests to the Hub, keep using get_session().
from huggingface_hub.utils import httpx
try:
...
except httpx.HTTPError:
...📚 Documentation: Utilities — The httpx module
🔒 Security
HfFileSystem.get() now validates remote filenames before writing anything locally. A server-side filename such as folder/..\..\outside.txt could previously escape the destination directory on Windows during a recursive download. The same check already protected hf_hub_download, snapshot_download and bucket sync. Unsafe filenames now raise ValueError on all platforms, including when downloading to an explicitly named file or a file object.
🖥️ CLI
- [CLI] Fix
hf buckets rm --recursivedeleting lexical siblings of the prefix: removinglogs/no longer toucheslogs_root.txtby @Wauplin in #4804 - [CLI] Don't let
hf updateself-upgrade a pip install on Windows: it now prints thepip install -Ucommand to run instead of leaving a broken install by @Wauplin in #4823 - [CLI] Raise explicit error for shell-script extensions on Windows by @hanouticelina in #4846
- [CLI] Fix truncated command descriptions in the CLI reference by @hanouticelina in #4849
🐛 Bug and typo fixes
- [hf_api] Make
CommitInfocopy-able and pickle-able by @Wauplin in #4822 - [LFS] Fix
SliceFileObj.__iter__yielding only the first 4MB chunk by @BenYang12 in #4844 - Fix dotenv parser truncating unquoted values containing
#(used byhf jobs --env-file) by @BenYang12 in #4842 - Keep single-quoted values verbatim in
load_dotenvby @AnishPatel526 in #4827
📖 Documentation
- [Docs] Clarify Jobs argument forwarding with
--and align examples by @davanstrien in #4809
🏗️ Internal
- Bump the actions group across 1 directory with 5 updates by @dependabot[bot] in #4798
- Post-release: bump version to 1.31.0.dev0 by @huggingface-hub-bot[bot] in #4806
[v1.30.0] Scheduled job filters, repo-aware revision pinning and more
🖥️ hf jobs scheduled ls can now filter by status, label and name
You can now filter scheduled jobs with --status (active/suspended), repeatable -l/--label key=value and --name (a shortcut for --label name=NAME), matching the flags hf jobs ls already had. On the Python side, HfApi.list_scheduled_jobs gains a labels argument.
Warning
-f/--filter on hf jobs scheduled ls is deprecated and no longer applied: it is accepted but ignored, with a warning pointing to the new flags. Migrate as shown below. The option itself will be removed in a future release.
hf jobs scheduled ls --all
hf jobs scheduled ls --status suspended
hf jobs scheduled ls --name hourly-task --label env=prod| before | after |
|---|---|
-f name=daily-report |
--name daily-report |
-f env=prod |
--label env=prod |
-f suspend!=true |
(default, or --status active) |
-f suspend=true |
--status suspended |
📚 Documentation: Jobs guide
- Align
hf jobs scheduled lsfiltering withhf jobs ls(--status/--label/--name) by @moon-bot-app[bot] in #4773
🔖 ResolvedRevision now pins the repo it was resolved for
ResolvedRevision didn't record which repo a commit hash was resolved against, so resolving "main" on two different repos returned the same commit oid. It now remembers its repo_id and repo_type: passed back for the same repo it is returned as is, for a different repo the requested revision is resolved again.
>>> from huggingface_hub import resolve_revision
>>> revision = resolve_revision("openai-community/gpt2")
# Same repo => returned as is
>>> resolve_revision("openai-community/gpt2", revision=revision) is revision
True
# Another repo => "main" is resolved again for that repo
>>> resolve_revision("openai-community/gpt2-medium", revision=revision).resolved
'6dcaa7a952f72f9298047fd5137cd6e4f05f41da'📚 Documentation: Manage cache — Pin a revision
🤖 Inference
- Do not require the
conversationaltag for chat-completion on hf-inference: any model with pipeline tagtext-generationorimage-text-to-textcan now be served, mirroring the updated provider-side rules by @hanouticelina in #4784
🐛 Bug and typo fixes
- [Download] Follow redirects between Hub hosts when resolving files, instead of failing with a misleading connection error (redirects to CDNs are still not followed, so LFS metadata is preserved and
Authorizationis never forwarded off-Hub) by @hanouticelina in #4739
📖 Documentation
- [Sandbox] Mark the Sandbox API as experimental across docstrings, guides, package reference and CLI help, and clarify that shared sandboxes are intended for workloads within the same trust boundary by @Wauplin in #4783
🏗️ Internal
- [Release] Refresh OpenCode model cache (
opencode models --refresh) before validatingRELEASE_NOTES_MODEL, fixing release CI failures on cold runners by @Wauplin in #4761 - [Release] Consolidate the HF CLI skill sync into the release workflow, removing the duplicate standalone sync workflow that opened two PRs on
huggingface/skillsper release by @hanouticelina in #4765 - [CI] The close-unscoped-community-prs workflow stays available for manual runs only: enforcement was enabled and then disabled again within this release by @hanouticelina and @Wauplin in #4766, #4800
- [CI] Fix torch_1.11 case-arm label to match the renamed matrix entry, restoring torch installation on that lane by @MohammedAlkindi in #4776
- Post-release: bump version to 1.30.0.dev0 by @huggingface-hub-bot[bot] in #4763