did you try this? is it relevant? the comment is from 9 years ago.

We do see repositories getting throttled at LLNL from GitHub, but I'm not sure how I'd test the load on GitHub's side.

I think it's worth erring on the side of caution here until we hear otherwise from GitHub. I looked at brew and it still looks like they're preferring full clones over shallow clones for taps.

In my experience the initial shallow fetch is significantly faster, and for updates I expect the same.

There is also the use case of CI with ephemeral environments where full clones make little sense.

In my experience the initial shallow fetch is significantly faster, and for updates I expect the same.

The initial fetch is faster, but shallow updates don't really work. (I thought they'd work but it turns out they don't after living with spack repo update for a few weeks now.)

If > 20 commits have happened since you last ran spack repo update it fails with an error about disconnected histories.
(TLDR; repo update right now is broken is and we should fix it before we release tomorrow.)

To correctly do shallow updates we'd have to modify the depth argument dynamically to be the number of commits since the user's HEAD and I believe that's what folks are talking about here, and that operation is computationally expensive on GitHub's side.

If we drop the depth argument all together on the second time we run spack repo update (what I'd kinda assume would do a dynamic depth) git appears to just do a full clone in the fetch.

For CI/CD use cases I agree it would be really nice to keep the shallow cloning, I just don't know how to do that yet.

(Edit: testing with blobless clones I'm seeing near identical download times and comparable disk usage.)

@alecbcs can we:

1. Keep the initial clone shallow
2. git fetch --unshallow before updating, if we notice an already-fetched repo is shallow?

spack blame already unshallows repos whenever it's run in order to get full blame history.

Doing this would:

1. Keep the fast initial clones (e.g. for CI and places where you'll never update)
2. Do a big fetch on your 2nd update (which I think is a reasonable predictor that you will do more updates)
3. Allow updates to succeed without disconnected histories
4. Ensure that this works for people who already cloned the package repo with spack and are having update issues.

Thoughts?

I've dropped the --filter=blobs:none in favor of @tgamblin's suggestion since we're still supporting users on RHEL7 with old versions of git that don't yet support blobless clones.

I've tested this on a shallow download and it successfully updated the repository by deepening it to a full clone.