perf: faster blobless git clones#290
Conversation
0d74bdd to
23ae397
Compare
|
I wonder if even a shallow fetch with depth 1 would be sufficient? This could result in an even bigger performance boost. There is a follow up post with some performance measurements: https://github.blog/2020-12-22-git-clone-a-data-driven-study-on-cloning-behaviors/ In my understanding (which may be wrong), poetry always does a Of course, using shallow fetch would be a more significant change compared to blobless clone. Probably, shallow fetch should not replace the existing |
|
It could work. And it could be better. However, that would be more drastic. For example, if poetry kept a real cache, it would be more useful with a blobless clone than with a shallow clone. This one enhancement is good and means practically no difference in current behavior. Actually, if all tests go green, is enough prove that this won't change anything deeply important. I think that can be considered as a separate future enhancement, but I don't see that as a blocker for this enhancement. OTOH, neither of these fix the problem explained in python-poetry/poetry#5188 (quadruplicated clones). Again, not a blocker for this enhancement IMHO. |
|
In the linked pip PR, the git version is checked. Maybe, we should also check the version. |
|
What is the lowest version supported here for git? |
|
Probably, this is not specified anywhere. |
23ae397 to
1fdc00b
Compare
|
Version checked. |
|
Kudos, SonarCloud Quality Gate passed!
|
|
@yajo Can you please rebase your branch in order to allow merging? Probably, you have to adapt the type hints. |
Cloning a repo with git can be really slow if the repo is big. Adding `--filter=blob:none` to the `git clone` command will make it really faster and slimmer because it will lazy-download blobs on checkout, while only getting commit metadata for whatever is not checked out. See "Blobless Clones" in https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ for a good explanation. Also interesting to see that pip 21.3 itself landed this enhancement: pypa/pip#9086. @moduon MT-83
|
I think it should be good now. |
|
Kudos, SonarCloud Quality Gate passed!
|
Cloning a repo with git can be really slow if the repo is big. Adding `--filter=blob:none` to the `git clone` command will make it really faster and slimmer because it will lazy-download blobs on checkout, while only getting commit metadata for whatever is not checked out.
Cloning a repo with git can be really slow if the repo is big. Adding `--filter=blob:none` to the `git clone` command will make it really faster and slimmer because it will lazy-download blobs on checkout, while only getting commit metadata for whatever is not checked out.








Cloning a repo with git can be really slow if the repo is big.
Adding
--filter=blob:noneto thegit clonecommand will make it really faster and slimmer because it will lazy-download blobs on checkout, while only getting commit metadata for whatever is not checked out.See "Blobless Clones" in https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ for a good explanation.
Also interesting to see that pip 21.3 itself landed this enhancement: pypa/pip#9086.
@moduon MT-83