Skip to content

GDAL: dependency variants#31197

Merged
scheibelp merged 15 commits intospack:developfrom
adamjstewart:packages/gdal2
Jul 13, 2022
Merged

GDAL: dependency variants#31197
scheibelp merged 15 commits intospack:developfrom
adamjstewart:packages/gdal2

Conversation

@adamjstewart
Copy link
Copy Markdown
Member

@adamjstewart adamjstewart commented Jun 20, 2022

This PR is an alternative to #30916. It still adds CMake support, but keeps the variants the way they were before.

  • Add GDAL 3.5.0
  • Remove GDAL 1
  • Add support for new CMake build system
  • Change defaults to build all recommended dependencies
  • Simplify Autotools flag handling
  • Determine version range for drivers

Tested the following versions:

  • GDAL 3.5
  • GDAL 3.4
  • GDAL 3.3
  • GDAL 3.2
  • GDAL 3.1
  • GDAL 3.0
  • GDAL 2.4
  • GDAL 2.3
  • GDAL 2.2
  • GDAL 2.1
  • GDAL 2.0

On the following platforms:

  • macOS 12.4 and Apple M1 Pro (arm64) with Apple Clang 13.1.6 (all versions)
  • macOS 10.15.7 and Intel Ivybridge (x86_64) with Apple Clang 12.0.0 (3.5)

With the following configurations:

  • Default variants (all versions)
  • Enable all language bindings (3.4, 3.5)
  • Enable all optional dependencies (3.5)
  • Disable all recommended dependencies (all versions)

External links:

The Autotools build system is no longer maintained, so anyone having build issues on old versions should switch to the CMake version.

Closes #30916
Closes #30809
Closes #30808
Closes #23943

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Jun 20, 2022

Hi @adamjstewart! I noticed that the following package(s) don't yet have maintainers:

  • brunsli

Are you interested in adopting any of these package(s)? If so, simply add the following to the package class:

    maintainers = ['adamjstewart']

If not, could you contact the developers of this package and see if they are interested? You can quickly see who has worked on a package with spack blame:

$ spack blame brunsli

Thank you for your help! Please don't add maintainers without their consent.

You don't have to be a Spack expert or package developer in order to be a "maintainer," it just gives us a list of users willing to review PRs or debug issues relating to this package. A package can have multiple maintainers; just add a list of GitHub handles of anyone who wants to volunteer.

@adamjstewart adamjstewart marked this pull request as draft June 20, 2022 04:54
This was referenced Jun 22, 2022
@adamjstewart adamjstewart reopened this Jun 23, 2022
@adamjstewart
Copy link
Copy Markdown
Member Author

Alright, this alternative PR is now ready for review!

Pinging the same folks as #30916: @Sinan81 @snehring @haralmha @rouault @amaji @payerle @michaelkuhn @iarspider @neilflood

This PR also adds GDAL 3.5.0 and CMake support, but does it in a completely different way. The biggest change is w.r.t. variants:

This is the same way the variants are in develop, and seems to be significantly easier to maintain. With #30916 we would probably still need to use the same flags for Autotools to tell the build system where to find the dependencies, and the exact driver -> CMake/Autotools flag is nonintuitive. For a comparison of the relative complexity, the build recipe in this PR is ~700 lines while the build recipe in #30916 is 1,000+ lines.

This PR also allows us to retain GDAL 2 support, which several packages still rely on. I also added a bunch of new variants for dependencies that happen to be in Spack. I might add more in the future once I package the dependencies.

Let me know if you prefer this PR or #30916. In terms of maintainability I think I'm definitely leaning towards this one.

@neilflood
Copy link
Copy Markdown
Contributor

"Easier to maintain" wins a lot of points, in my view. This does look better, and I am most impressed that you were able to include both the autotools and cmake builds in the one recipe, and hence keep all the older versions. Legend!

I confess I got a bit lost about where you are up to with the default set of drivers (as discussed in #30916). Could you clarify where you are aiming with that?

@adamjstewart
Copy link
Copy Markdown
Member Author

I am most impressed that you were able to include both the autotools and cmake builds in the one recipe, and hence keep all the older versions.

To clarify, both #30916 and #31197 include both Autotools and CMake build system support. The difference is that #30916 uses Autotools flags that weren't added until GDAL 3.0, so that PR only supports GDAL 3.0–3.4 (Autotools) and 3.5+ (CMake).

I confess I got a bit lost about where you are up to with the default set of drivers (as discussed in #30916). Could you clarify where you are aiming with that?

In the current recipe (develop), only required dependencies were added by default, you had to enable all variants yourself. In both #30916 and #31197, all required AND recommended dependencies are enabled by default. There are still several dozen optional dependencies that you have to manually enable if you want them. You can see the default=(True|False) in the variant definitions to see which are enabled by default. I think enabling recommended dependencies by default is a good compromise. We want to keep build times as short as possible and prevent issues where certain dependencies don't build on all systems, while still keeping the default build relevant to most users.

@neilflood
Copy link
Copy Markdown
Contributor

That all makes sense. Thank you for the clarifications. :-)

@adamjstewart adamjstewart marked this pull request as ready for review July 5, 2022 19:04
@adamjstewart
Copy link
Copy Markdown
Member Author

Is anyone else able to review this PR?

@scheibelp scheibelp self-assigned this Jul 12, 2022
Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one request for adding a comment (related to "overriding" of cmake phase) but otherwise this looks good.

That's not essential though: so if you're strapped for time then this is good as-is (I don't see any issues with the logic).

@scheibelp scheibelp merged commit 752697a into spack:develop Jul 13, 2022
@adamjstewart adamjstewart deleted the packages/gdal2 branch July 13, 2022 19:39
bhatiaharsh pushed a commit to bhatiaharsh/spack that referenced this pull request Aug 8, 2022
Add support for CMake builds while preserving autotools support
for older versions of GDAL

* Add GDAL 3.5.0
* Remove GDAL 1
* Add support for new CMake build system
* Change defaults to build all recommended dependencies
* Simplify Autotools flag handling
* Determine version range for drivers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Installation issue: gdal-3.4.2 on NOAA RDHPCS Gaea (Cray) Installation issue: gdal

3 participants