Add compilers paths for dependents of intel-parallel-studio by DaanVanVugt · Pull Request #2980 · spack/spack

DaanVanVugt · 2017-02-01T15:46:15Z

davydden

looks good!

zzzoom · 2017-02-01T18:13:07Z

Is this just for Intel MPI or the whole Intel toolchain?
mpi{icc,icpc,ifort} will always run intel compilers, but if you want to use Intel MPI with any other compiler you should set I_MPI_CC and run mpicc.

davydden · 2017-02-01T18:21:29Z

mpi{icc,icpc,ifort} will always run intel compilers, but if you want to use Intel MPI with any other compiler you should set I_MPI_CC and run mpicc.

that's a valid point. We should also make sure Spack's compiler wrappers are used underneath. For openmpi this is done via

        spack_env.set('OMPI_CC', spack_cc)
        spack_env.set('OMPI_CXX', spack_cxx)
        spack_env.set('OMPI_FC', spack_fc)
        spack_env.set('OMPI_F77', spack_f77)

in setup_dependent_environment.

@zzzoom i never used Intel MPI, so does it always have mpicc in bins?

zzzoom · 2017-02-01T18:26:12Z

I can see both mpicc and mpiicc in my IPSXE 2016u4 mpi/bin64 directory, dunno how long it has been like that for.

davydden · 2017-02-01T18:28:18Z

@lee218llnl if I am not mistaken, you contributed intel packages, do you know about differences between mpicc and mpiicc and how we should add these wrappers in Spack?

davydden · 2017-02-01T19:00:40Z

@Exteris @zzzoom it looks like we can use mpicc with intel compilers as well:

Note that if you want to use mpif90/mpicc/etc (eg for existing configure scripts/Makefiles, etc) with the Intel compilers, one need only modify environment variables to point to the appropriate compilers: eg,
export I_MPI_CC=icc
export I_MPI_CXX=icpc
export I_MPI_F77=ifort
export I_MPI_F90=ifort

from https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/288354

So maybe one just need to add

spack_env.set('I_MPI_CC', spack_cc)
spack_env.set('I_MPI_CXX', spack_cxx)
spack_env.set('I_MPI_F77', spack_fc)
spack_env.set('I_MPI_F90', spack_f77)

together with standard wrapper names (i.e. mpicc).

lee218llnl · 2017-02-01T19:29:35Z

@davydden it looks like you answered your own question. By default the mpicc, mpicxx, mpif, etc. scripts will use the GNU compilers unless you set the I_MPI_* variables. Note you should also add:

spack_env.set('I_MPI_FC', spack_fc)

davydden · 2017-02-01T19:33:23Z

@lee218llnl thanks for confirming.

@Exteris would you mind to update the PR accordingly?

DaanVanVugt · 2017-02-02T08:01:14Z

Updated.
I need to stress for this PR that I have not tested this with a spack-installed intel-parallel-studio, but only with intel-parallel-studio loaded from a module. Perhaps someone who installs their own IPSXE can try?

davydden · 2017-02-02T08:06:27Z

var/spack/repos/builtin/packages/intel-parallel-studio/package.py

            # file if Intel MPI is the dominant, or only, MPI on a system.
            run_env.set('I_MPI_ROOT', join_path(self.prefix, 'impi'))
+            # Set mpi environment variables
+            spack_env.set('I_MPI_CC', spack_cc)


I am not sure but in openmpi these things are done in def setup_dependent_environment(self, spack_env, run_env, dependent_spec):. see the docs http://spack.readthedocs.io/en/latest/packaging_guide.html?highlight=setup_dependent_environment
setup_environment appears to be mostly modules related http://spack.readthedocs.io/en/latest/module_file_support.html?highlight=setup_environment

You are probably right, I'll alter it.
What is the function signature? In the docs I see both:

def setup_dependent_environment(self, module, spec, dep_spec): """Dependencies of Qt find it using the QTDIR environment variable.""" os.environ['QTDIR'] = self.prefix

The arguments to this function are:

module: the module of the dependent package, where global properties can be assigned. spec: the spec of the dependency package (the one the function is called on). dep_spec: the spec of the dependent package (i.e. dep_spec depends on spec).

and

def setup_dependent_environment(self, spack_env, run_env, extension_spec):

The difference in names confuses me.

i would take the same one which is used in openmpi:

def setup_dependent_environment(self, spack_env, run_env, dependent_spec):

I fixed the inconsistencies in the documentation in #3351. You should use:

setup_dependent_environment(self, spack_env, run_env, dependent_spec):

davydden · 2017-03-18T18:45:01Z

@Exteris what's the status of this PR? would you mind fixing the minor issue related to setup_dependent_environment?

tgamblin · 2017-03-18T19:16:52Z

@weijianwen: I think you could update this PR slightly to fix your issue.

davydden · 2017-04-19T11:57:45Z

@Exteris ping.

davydden · 2017-04-21T15:26:30Z

var/spack/repos/builtin/packages/intel-parallel-studio/package.py

+            elif os.path.isdir(join_path(self.prefix, 'bin64')):
+                bindir = join_path(self.prefix, 'bin64')
+            else:
+                raise "No suitable bindir found"


in another PR, @adamjstewart said that this should probably be raise RuntimeError('No suitable bindir found')

davydden · 2017-04-21T15:45:59Z

superseded by #3905

Add compilers paths for dependents of intel-parallel-studio

67dcbed

davydden approved these changes Feb 1, 2017

View reviewed changes

Add I_MPI_CC etc variables for intel-parallel-studio

6c7169c

davydden reviewed Feb 2, 2017

View reviewed changes

davydden added the WIP label Feb 2, 2017

davydden mentioned this pull request Mar 18, 2017

spec['mpi'].mpicc doesn't work for MPI in intel-parallel-studio #3487

Closed

davydden mentioned this pull request Apr 21, 2017

add Intel mpi package and add MPI wrappers to Intel parallel studio #3905

Merged

2 tasks

davydden reviewed Apr 21, 2017

View reviewed changes

davydden closed this Apr 21, 2017

This was referenced Jun 27, 2017

Requested changes related to mumps compilation #4614

Merged

HDF5 installation with Intel compiler 17.0.4 and Intel MPI 2017.3 #4630

Closed

Conversation

DaanVanVugt commented Feb 1, 2017 • edited by davydden Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davydden left a comment

Choose a reason for hiding this comment

Uh oh!

zzzoom commented Feb 1, 2017

Uh oh!

davydden commented Feb 1, 2017

Uh oh!

zzzoom commented Feb 1, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davydden commented Feb 1, 2017

Uh oh!

davydden commented Feb 1, 2017

Uh oh!

lee218llnl commented Feb 1, 2017

Uh oh!

davydden commented Feb 1, 2017

Uh oh!

DaanVanVugt commented Feb 2, 2017

Uh oh!

davydden Feb 2, 2017

Choose a reason for hiding this comment

Uh oh!

DaanVanVugt Feb 2, 2017

Choose a reason for hiding this comment

Uh oh!

davydden Feb 2, 2017

Choose a reason for hiding this comment

Uh oh!

adamjstewart Mar 18, 2017

Choose a reason for hiding this comment

Uh oh!

davydden commented Mar 18, 2017

Uh oh!

tgamblin commented Mar 18, 2017

Uh oh!

davydden commented Apr 19, 2017

Uh oh!

davydden Apr 21, 2017

Choose a reason for hiding this comment

Uh oh!

davydden commented Apr 21, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

DaanVanVugt commented Feb 1, 2017 •

edited by davydden

Loading

zzzoom commented Feb 1, 2017 •

edited

Loading