I created a PR that is motivated by experience using hf buckets sync in trackio, where downloading a bucket locally took a very long time. I've fixed it trackio but wondering whether we should try to optimize this upstream. In this PR, we:

• avoid walking the full local destination tree when syncing bucket -> local without --delete
• stat only the remote-relative paths that can affect the plan, which keeps download sync fast even if the destination lives under a large directory like ~/.cache/huggingface/
• keep the full local walk for --delete, where local-only files still need to be discovered

The main reasoning here is that local-only files are irrelevant to the sync plan unless we intend to delete them. For regular download syncs, the plan only depends on remote files and whether each corresponding local path already exists / differs, so a targeted stat is enough. But leaving this as a draft PR as this could, in other cases, lead to worse performance, i.e. if the remote bucket is very large and the local destination is relatively small, doing one stat per remote path can be slower than a single full local walk. This might actually be more common, although this was not the case for trackio. Also, according to Codex, there are also some filesystem edge cases where targeted stat calls may behave a bit differently from the previous os.walk-based scan