py-netcdf4: enable non-MPI build per variant, even if netcdf-c was built with MPI#48694
Conversation
skosukhin
left a comment
There was a problem hiding this comment.
I'm fine with moving all @:1.6 versions under with default_args(deprecated=True):.
|
This is the full patch that seems to allow for --- a/include/netCDF4.pxi
+++ b/include/netCDF4.pxi
@@ -7,2 +7,2 @@
-cdef extern from "H5public.h":
- ctypedef int herr_t
+ctypedef int herr_t
+cdef extern from *:
--- a/setup.py
+++ b/setup.py
@@ -395 +395 @@
- has_parallel_support = check_has_parallel_support(inc_dirs)
+ has_parallel_support = False
Here is the patch with more context.--- a/include/netCDF4.pxi
+++ b/include/netCDF4.pxi
@@ -4,8 +4,8 @@
ctypedef long ptrdiff_t
# hdf5 version info.
-cdef extern from "H5public.h":
- ctypedef int herr_t
+ctypedef int herr_t
+cdef extern from *:
int H5get_libversion( unsigned int *majnum, unsigned int *minnum, unsigned int *relnum ) nogil
cdef extern from *:
--- a/setup.py
+++ b/setup.py
@@ -392,7 +392,7 @@
(netcdf_lib_version > "4.4" and netcdf_lib_version < "4.5"):
has_cdf5_format = True
- has_parallel_support = check_has_parallel_support(inc_dirs)
+ has_parallel_support = False
has_has_not = "has" if has_parallel_support else "does not have"
print(f"netcdf lib {has_has_not} parallel functions")
We apply the patch as follows: with when("@1.7:~mpi"):
patch("disable_parallel_support.patch", when="^netcdf-c+mpi")
patch("disable_parallel_support.patch", when="^hdf5+mpi")It might be enough to apply the patch only when And Wow! It looks like all that is just to emit an obsolete warning under a broken condition. |
|
I updated the patch and package as you suggested, and I built |
|
It would be good to not let this stagnant, is there anything I can test to help move this along? |
|
Friday evening is for sure not the best time for a reminder like this :) But thank you, I will try my best to take a look at this on Monday. |
var/spack/repos/builtin/packages/py-netcdf4/disable_parallel_support.patch
Outdated
Show resolved
Hide resolved
Co-authored-by: Sergey Kosukhin <[email protected]>
|
@skosukhin Sorry that I apparently missed your request for changes three weeks ago. I applied the suggestions and built py-netcdf4 ~mpi with netcdf-c +mpi successfully on my laptop (gcc@13). |
|
@climbfuji are you just waiting on a review for this? |
For the record, Spack still trims the dependencies of external packages |
var/spack/repos/builtin/packages/py-netcdf4/disable_parallel_support.patch
Outdated
Show resolved
Hide resolved
var/spack/repos/builtin/packages/py-netcdf4/disable_parallel_support.patch
Outdated
Show resolved
Hide resolved
…-pick changes from spack#48694, change numpy@2 dependency to @1.26
|
@skosukhin I had to revert part of the patch in commit 8976169 that you contributed a while ago. With that patch, I got the errors reported in #48694 (comment). Would you be able to review again and merge if it looks ok? |
|
@climbfuji what is the value of this MR then? What is the spec that you couldn't install and that can be installed with these changes? |
|
|
It turns out that we cannot even |
I am still on the last commit before the compilers-as-nodes changes were merged (three weeks-ish ago) and |
|
@skosukhin I have also had issues with mpi on the recent py-netcdf As it ingests a header that has mpi (when +mpi) I think this should be merged and then I can rebase #50075 on top of this PR. |
skosukhin
left a comment
There was a problem hiding this comment.
Please, fix the PR description.
and when netCDF was built with +mpi but hdf5 was built with ~mpi
- It's not possible to have
netcdf-c+mpi ^hdf5~mpi. - We can't have
netcdf-c~mpi ^hdf5+mpiafter you removed the respective part of the patch.
Co-authored-by: Sergey Kosukhin <[email protected]>
Done, thanks. |
|
@alalazo could you merge this, please? |
…ilt with MPI (spack#48694) * Add patch for py-netcdf4 so that we can build py-netcdf4 with ~mpi when netCDF was built with +mpi * Address reviewer requests for py-netcdf4. Add conflict for 'pynetcdf4~mpi ^netcdf-c~mpi ^hdf5+mpi' * Make py-netcdf4~mpi ^netcdf-c~mpi ^hdf5+mpi work * Apply suggestions from code review Co-authored-by: Sergey Kosukhin <[email protected]> * Update var/spack/repos/builtin/packages/py-netcdf4/disable_parallel_support.patch * Apply suggestions from code review Co-authored-by: Sergey Kosukhin <[email protected]> --------- Co-authored-by: Sergey Kosukhin <[email protected]>
Description
Allow building
py-netcdf4 ~mpiwhennetcdf-cwas built with+mpi. The patch overrides the auto-detect feature (has_parallel_support) insetup.py. The logic insetup.pychanged between 1.6.5 and 1.7.1, therefore this patchonly works for versions 1.7.1 and later.
Unidata/netcdf4-python#1389
@Chrismarsh FYI