Skip to content

Close #1683 Apply ADIOS 1.10.0 Patch (HDF5 1.10+)#1699

Merged
tgamblin merged 2 commits intospack:developfrom
ax3l:fix-adiosHDF5110
Sep 21, 2016
Merged

Close #1683 Apply ADIOS 1.10.0 Patch (HDF5 1.10+)#1699
tgamblin merged 2 commits intospack:developfrom
ax3l:fix-adiosHDF5110

Conversation

@ax3l
Copy link
Copy Markdown
Member

@ax3l ax3l commented Sep 2, 2016 •
edited
Loading

Close #1683: Apply the patch ornladios/ADIOS@3b21a8a41509 for all released ADIOS version up to the latest 1.10.0 which will break when used with HDF5 1.10+.

The bug is already fixed upstream in ADIOS.

Other Changes

  • fix doc strings: --with-hdf5 (and --with-netcdf) are not transports but serial file converters
  • remove bp2ncd tool: depends on netcdf 3 and is not essential
  • hdf5:
    • since spec['hdf5+mpi'].prefix does not exist, we only add one +hdf5 variant, its building
      • the parallel transport --with-phdf5
      • the serial converter --with-hdf5, but also build by --with-phdf5
      • building the serial converter alone is intentionally not possible this way and reduces confusion
  • add: develop version as described spack_workflows.rst (the ADIOS development branch is master)

see: official manual, section 2.3

Tests

  • old fail
# unpatched version
$ spack install adios +hdf5 # implies currently hdf5 1.10.0-patch1
==> No patches needed for adios
==> Building adios
==> Error: Command exited with status 2:
'make' '-j4'
[...]
  >> 132          make()
[...]
  • new: only apply patch when necessary
# old HDF5 version, latest (1.10.0) ADIOS version
$ spack install adios +hdf5 ^[email protected]
[...]
==> Created stage in [...]/var/spack/stage/hdf5-1.8.16-[...]
==> No patches needed for hdf5
==> Building hdf5
==> Successfully installed hdf5
[...]
No patches needed for adios
==> Building adios
==> Warning: Patched overlong shebang in [...]opt/spack/linux-debian8-x86_64/gcc-4.9.2/adios-1.10.0-[...]/bin/gpp.py
==> Successfully installed adios
  • new: apply patch to fix
# latest versions
$ spack install adios +hdf5 # implies currently hdf5 1.10.0-patch1
[...]
==> Created stage in [...]var/spack/stage/hdf5-1.10.0-patch1-[...]
==> No patches needed for hdf5
==> Building hdf5
==> Successfully installed hdf5
[...]
==> Created stage in [...]/var/spack/stage/adios-1.10.0-[...]
==> Applied patch adios_1100.patch
==> Building adios
==> Warning: Patched overlong shebang in [...]opt/spack/linux-debian8-x86_64/gcc-4.9.2/adios-1.10.0-x6woxnqfn2gzu6smq5xgxtqrglpquowd/bin/gpp.py
==> Successfully installed adios

@ax3l ax3l changed the title Close #1683 Apply ADIOS 1.10.0 Patch Close #1683 Apply ADIOS 1.10.0 Patch (HDF5 1.10+) Sep 2, 2016
@eschnett
Copy link
Copy Markdown
Contributor

eschnett commented Sep 2, 2016

Note that the original issue also mentions that Adios might require hdf5 +mpi; if we close this issue, then we should address that dependency as well.

@ax3l
Copy link
Copy Markdown
Member Author

ax3l commented Sep 2, 2016 •
edited
Loading

I just checked the manual, section 2.3 again and realized it's fine so far. Currently we are only installing serial file converter tools, not the parallel transports that would depend on hdf5+mpi.

planning the hdf5+mpi option as a feature enhancement in a follow-up.

ax3l
ax3l reviewed Sep 2, 2016
View reviewed changes
var/spack/repos/builtin/packages/adios/package.py Outdated
depends_on('netcdf', when='+netcdf')
# optional serial file converters
depends_on('[email protected]:', when='+hdf5')
depends_on('netcdf@3', when='+netcdf')
Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 2, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ADIOS Manual, section 2.3.10: NetCDF-3.
Currently not available in spack. imho, we can remove this converter option for now until it's updated to netcdf 4

Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed now

@ax3l
Copy link
Copy Markdown
Member Author

ax3l commented Sep 2, 2016 •
edited
Loading

can't finish the tests due to #1701

@ax3l
Copy link
Copy Markdown
Member Author

ax3l commented Sep 2, 2016

hdf5 install options simplified to be only hdf5 +mpi, having two builds is too confusing

davydden
davydden reviewed Sep 2, 2016
View reviewed changes
var/spack/repos/builtin/packages/adios/package.py Outdated
depends_on('hdf5', when='+hdf5')
depends_on('netcdf', when='+netcdf')
# optional transports & file converters
depends_on('[email protected]: +mpi', when='+hdf5')
Copy link
Copy Markdown
Member

@davydden davydden Sep 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+hdf5+mpi?

Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 2, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did not work without the space somehow, can retry
(started clean rebuild... will take a while)

Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 2, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works, must have been an artefact. will amend :)
(also have to fix flake8)

@ax3l
Copy link
Copy Markdown
Member Author

ax3l commented Sep 3, 2016 •
edited
Loading

@davydden @eschnett all problems of #1683 addressed, fixed and tested :)

@davydden
Copy link
Copy Markdown
Member

davydden commented Sep 3, 2016

looks good.

ax3l added 2 commits September 12, 2016 16:09
@ax3l
Close spack#1683 Apply ADIOS 1.10.0 Patch
30234cd 
Also correct version dependencies and comments.
@ax3l
Clean Up ADIOS
94cb4d9 
- add develop
- simplify HDF5 options (parallel only)
- remove optional netCDF (not fully v4)
@ax3l
Copy link
Copy Markdown
Member Author

ax3l commented Sep 12, 2016

rebased against latest develop, pls merge me :)

@ax3l ax3l mentioned this pull request Sep 13, 2016
Merged
@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Sep 21, 2016

@ax3l: thanks!

@tgamblin tgamblin merged commit d3e04b3 into spack:develop Sep 21, 2016
@ax3l ax3l deleted the fix-adiosHDF5110 branch September 28, 2016 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Adios does not build with HDF5 1.10

4 participants

@ax3l @eschnett @davydden @tgamblin