Skip to content

Comments

g.extension: sparse checkout with history for fast commit log access#3051

Merged
ninsbl merged 2 commits intoOSGeo:mainfrom
ninsbl:g_extension_log
Jun 15, 2023
Merged

g.extension: sparse checkout with history for fast commit log access#3051
ninsbl merged 2 commits intoOSGeo:mainfrom
ninsbl:g_extension_log

Conversation

@ninsbl
Copy link
Member

@ninsbl ninsbl commented Jun 13, 2023

git -log -1 is slow with sparse checkout if --filter=tree:0 is used because history is not checked out. Cloning is similarly fast with --filter=blob:none but then the history is checked out too and git log -1 should be fast again.

Please try e.g.:

dirs="src/raster/r.centroids src/imagery/i.pysptools.unmix"
cd /tmp
time (git clone -q --no-checkout --branch=grass8 --filter=blob:none https://github.com/osgeo/grass-addons
cd grass-addons
git sparse-checkout set $dirs
git checkout grass8)

cd grass-addons
for dir in $dirs
do
  time git log --first-parent -1 $dir
  
done
cd ..
rm -rf grass-addons

Fixes #2979

I marked this as blocker as 30 seconds wait time per addon is a pretty bad user experience. Feel free to remove the label if you disagree...

@ninsbl ninsbl added bug Something isn't working blocker Blocking a release Python Related code is in Python backport to 8.3 backport to 7.8 PR needs to be backported to release branch 7.8 labels Jun 13, 2023
@ninsbl ninsbl added this to the 8.3.0 milestone Jun 13, 2023
@ninsbl ninsbl requested a review from tmszi June 13, 2023 20:53
Copy link
Member

@tmszi tmszi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and works as expected.

This code line required changes too.

"--filter=tree:0",

Small note:

With blob:none change size of downloaded data from 2.6M to 6.5M.

@ninsbl
Copy link
Member Author

ninsbl commented Jun 14, 2023

Tested and works as expected.

This code line required changes too.

"--filter=tree:0",

Thanks for testing and pointing out the missing change in mkhtml.py... Addressed in 48a86df.

@ninsbl ninsbl changed the title g.extension: sparse checkout with history fast commit log access g.extension: sparse checkout with history for fast commit log access Jun 15, 2023
@ninsbl ninsbl merged commit 0d6124b into OSGeo:main Jun 15, 2023
@landam landam modified the milestones: 8.3.0, 8.4.0 Jun 21, 2023
ninsbl added a commit to ninsbl/grass that referenced this pull request Jun 21, 2023
@ninsbl
Copy link
Member Author

ninsbl commented Jun 21, 2023

Cannot be backported to 8.2 directly, should be added to a bunch of g.extension-fixes that should be backported...

ninsbl added a commit that referenced this pull request Jun 21, 2023
@petrasovaa petrasovaa removed the backport to 7.8 PR needs to be backported to release branch 7.8 label Aug 11, 2023
landam pushed a commit to landam/grass that referenced this pull request Oct 25, 2023
neteler pushed a commit to nilason/grass that referenced this pull request Nov 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocker Blocking a release bug Something isn't working Python Related code is in Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] g.extension test often times out in CI

4 participants