Skip to content

Comments

CI: Refactor download of source code for rasterio libs#3493

Merged
snowman2 merged 9 commits intorasterio:mainfrom
w8sl:refactor_downloads_GDAL3.12
Dec 22, 2025
Merged

CI: Refactor download of source code for rasterio libs#3493
snowman2 merged 9 commits intorasterio:mainfrom
w8sl:refactor_downloads_GDAL3.12

Conversation

@w8sl
Copy link
Contributor

@w8sl w8sl commented Dec 20, 2025 •
edited
Loading

  • Separate downloads from compilation
  • Sort URL links in one place
  • Use a unified fetch_untar bash script with customizable retry and SHA-256 check logic
  • Confirm successful downloads within 3-4 minutes
  • Verbose all downloads. Show SHA256 hash for all libraries
  • Simplify adding SHA-256 hash check (applicable to all libraries if needed)
  • Unchanged URL links and build scripts, reformatted with shfmt
  • Windows build unchanged, managed by vcpkg

@snowman2 snowman2 added this to the 1.5.0 milestone Dec 22, 2025
snowman2
snowman2 approved these changes Dec 22, 2025
View reviewed changes
Copy link
Member

@snowman2 snowman2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @w8sl 👍

@snowman2 snowman2 merged commit cdf3a22 into rasterio:main Dec 22, 2025
33 checks passed
@w8sl
Copy link
Contributor Author

w8sl commented Dec 22, 2025

Thanks @snowman2 for implementing!

@w8sl
Copy link
Contributor Author

w8sl commented Dec 23, 2025

rasterio/.github/workflows/build-wheels.yaml

Line 251 in daa777b

needs: [build_wheels, build_sdist]

maybe:

needs: [build_wheels, build_sdist, test_wheels]

?

@snowman2
Copy link
Member

snowman2 commented Dec 23, 2025

rasterio/.github/workflows/build-wheels.yaml

Line 251 in daa777b

needs: [build_wheels, build_sdist]

maybe:

needs: [build_wheels, build_sdist, test_wheels]

?

Yes. If you wouldn't mind making a PR, that would be helpful.

@w8sl
Copy link
Contributor Author

w8sl commented Dec 23, 2025 •
edited
Loading

rasterio/.github/workflows/build-wheels.yaml

Line 251 in daa777b

needs: [build_wheels, build_sdist]

maybe:

needs: [build_wheels, build_sdist, test_wheels]

?

Yes. If you wouldn't mind making a PR, that would be helpful.

I have seen error with wrong jpeg library on macOS15 before (arm64 but the same error). Looks like they have updated macOS Intel runner. I will try to refresh memory and find a fix. But this runner is a bit slow, even for reading logs.

@w8sl
Copy link
Contributor Author

w8sl commented Dec 26, 2025

There is a conflict between jpeg and jpeg-turbo on macOS x86_64. It can be solved by removing all traces of jpeg-turbo or by switching to jpeg-turbo.
JPEG-TURBO is de-facto standard: vcpg uses it, Homebrew uses it, GDAL wants it for 12bit jpeg compatibility. It is 2,5 x faster when building GeoTIFFs with jpeg compression. I am coming with PR. No idea what happened with Windows failures. random error or related to latest commit?

@w8sl
Copy link
Contributor Author

w8sl commented Dec 26, 2025 •
edited
Loading

Windows tests are passing from a copy of PR3493 branch: https://github.com/w8sl/rasterio/actions/runs/20522926060

@w8sl w8sl deleted the refactor_downloads_GDAL3.12 branch January 7, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@snowman2 snowman2 snowman2 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.5.0

Development

Successfully merging this pull request may close these issues.

2 participants

@w8sl @snowman2