Skip to content

libSplash: Add Spack Package#1667

Merged
tgamblin merged 2 commits intospack:developfrom
ax3l:topic-libsplash
Aug 30, 2016
Merged

libSplash: Add Spack Package#1667
tgamblin merged 2 commits intospack:developfrom
ax3l:topic-libsplash

Conversation

@ax3l
Copy link
Copy Markdown
Member

@ax3l ax3l commented Aug 30, 2016

Adds a package for libSplash, a high-level library around serial and parallel HDF5 for regular grids and particle data sets.

libSplash aims at developing a HDF5-based I/O library for HPC
simulations. It is created as an easy-to-use frontend for the
standard HDF5 library with support for MPI processes in a cluster
environment. While the standard HDF5 library provides detailed
low-level control, libSplash simplifies tasks commonly found in
large-scale HPC simulations, such as iterative computations
and MPI distributed processes.

libSplash is a dependency for PIConGPU, an open-source, many-core, fully-relativistic particle-in-cell code and further software developed at Helmholz-Zentrum Dresden - Rossendorf.

libSplash builds in two versions, one without MPI writing domain-decomposed posix-style HDF5 files per process and one (default) with MPI and MPI-I/O ("parallel HDF5") support aggregating into a single file per MPI communicator.

libSplash is used in conjunction with openPMD, see also github.com/openPMD/.

Dependency Graph

$ spack graph libsplash+mpi
o  libsplash
|\
| |\
| o |  hdf5
|/| | 
| o |  zlib
|  /
o |  mpi
 /
o  cmake

$ spack graph --concretize libsplash+mpi
o  libsplash
|\
| |\
| | o  cmake
| | |\
| | o |  openssl
| o | |  hdf5
|/| | | 
| |/ /
| o |  zlib
|  /
o |  openmpi
| o  ncurses
| 
o  hwloc
o  libpciaccess
o  libtool
o  m4
o  libsigsegv

$ spack graph libsplash
o  libsplash
o  cmake

$ spack graph --concretize libsplash
o  libsplash
|\
| |\
| | o  cmake
| | |\
| | o |  openssl
| o | |  hdf5
|/| | | 
| |/ /
| o |  zlib
|  /
o |  openmpi
| o  ncurses
| 
o  hwloc
o  libpciaccess
o  libtool
o  m4
o  libsigsegv

$ spack graph libsplash~mpi
o  libsplash
|\
o |  hdf5
o |  zlib
 /
o  cmake

$ spack graph --concretize libsplash~mpi
o  libsplash
|\
| o  cmake
| |\
| o |  openssl
o | |  hdf5
|/ /
o |  zlib
 /
o  ncurses

# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this empty line.

Adds a package for
[libSplash](https://github.com/ComputationalRadiationPhysics/libSplash),
a high-level library around serial and parallel HDF5 for regular
grids and particle data sets.

```
libSplash aims at developing a HDF5-based I/O library for HPC
simulations. It is created as an easy-to-use frontend for the
standard HDF5 library with support for MPI processes in a cluster
environment. While the standard HDF5 library provides detailed
low-level control, libSplash simplifies tasks commonly found in
large-scale HPC simulations, such as iter- ative computations
and MPI distributed processes.
```

libSplash is a dependency for [PIConGPU](http://picongpu.hzdr.de),
an open-source, many-core, fully-relativistic particle-in-cell
code and further software developed at
[Helmholz-Zentrum Dresden - Rossendorf](https://www.hzdr.de).

libSplash builds in two versions, one without MPI writing
domain-decomposed posix-style HDF5 files per process and one
(default) with MPI and MPI-I/O ("parallel HDF5") support
aggregating into a single file per MPI communicator.

libSplash is used in conjunction with
[openPMD](http://openPMD.org), see also
[github.com/openPMD/](https://github.com/openPMD/).
@ax3l ax3l force-pushed the topic-libsplash branch from 167f437 to 3aaa077 Compare August 30, 2016 10:42
@davydden
Copy link
Copy Markdown
Member

LGTM


depends_on('cmake', type='build')
depends_on('[email protected]:', when='~mpi')
depends_on('[email protected]:+mpi', when='+mpi')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll find that if you try running something like:
spack spec libsplash ^[email protected]
It will complain that libsplash does not depend on hdf5. This is a bug in Spack's dependency resolution. A workaround for this is to tell it to always depend on hdf5:

depends_on('[email protected]:')
depends_on('hdf5+mpi', when='+mpi')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to expand on @adamjstewart comment, spack will make a union of dependencies, i.e. [email protected]: + hdf5+mpi = hdf5:1.8.6:+mpi, that's why it works.

p.s. hopefully there is a unit test to check this feature as there are more and more packages which rely on this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

I was wondering because spack graph libsplash did not show hdf5 but spack graph --concretize libsplash does in the list...

will update with your suggestion, thank you both!

Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamjstewart in the latest develop updates, this now fails with:

$ spack install libsplash
==> Error: Invalid spec: '[email protected]%[email protected]+cxx~debug+fortran~mpi+shared~szip~threadsafe arch=linux-debian8-x86_64'. Package hdf5 requires variant +mpi, but spec asked for ~mpi

weird, since+mpi is default=True and

$ spack install libsplash +mpi

and

$ spack install libsplash -mpi

and

$ spack install libsplash ~mpi

succeeds... any idea?

Copy link
Copy Markdown
Member Author

@ax3l ax3l Sep 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamjstewart was the dependency resolution bug fixed and I should add the when='~mpi' now? :)

(seems to fix it the install; informative spack spec libsplash ^[email protected] fails as described above)

@adamjstewart
```
think you'll find that if you try running something like:
spack spec libsplash ^[email protected]
It will complain that libsplash does not depend on hdf5.
This is a bug in Spack's dependency resolution. A workaround
for this is to tell it to always depend on hdf5.
```

@davydden
```
to expand on @adamjstewart comment, spack will make a union
of dependencies,
i.e. [email protected]: + hdf5+mpi = hdf5:1.8.6:+mpi, that's why it works.
```

thank you for the hint!
@tgamblin tgamblin merged commit fa99b76 into spack:develop Aug 30, 2016
@tgamblin
Copy link
Copy Markdown
Member

Thanks!

@ax3l ax3l deleted the topic-libsplash branch August 30, 2016 18:12
ax3l added a commit to ax3l/spack that referenced this pull request Sep 4, 2016
the dependency bug mentioned in
    spack#1667 (comment)

seems to be fixed now, and is even causing a bug when not rewritten
    spack#1667 (comment)

This fixes it.

- use any version of hdf5 if no mpi is requested
- enfore "parallel" version of hdf5 if mpi is requested
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants