initial package by lee218llnl · Pull Request #796 · spack/spack

lee218llnl · 2016-04-18T21:57:31Z

All,

Here is my initial implementation of the Intel compiler package, addressing #761. It currently distinguishes between the "cluster", "professional", and "composer" editions. This whole thing is admittedly a bit of a mess, so feedback is much appreciated, particularly w.r.t. how versions and variants are being handled. A lot of the mess is due to flexibility I tried adding and we can certainly clean things up by just setting components to "ALL" and not allow the removal of the libraries. I know here at LLNL, we don't care much about daal and ipp, and both take up a lot of disk space.

The package will need to be updated to use the license features of #553 and in the meantime you will have to set your license_path appropriately.

adamjstewart · 2016-04-19T15:10:58Z

var/spack/repos/builtin/packages/intel/package.py

+    variant('tools', default=True, description="Install the Intel Advisor, VTune Amplifier, and Inspector tools")
+
+    provides('mpi', when='@cluster:+mpi')
+    provides('mkl', when='+mkl')


I think it would be better to provide blas, lapack, and a new virtual package called fft. The atlas package similarly provides blas and lapack.

I think that sounds reasonable, but how well would that work since the library names are different? I suppose we could create symlinks with the standard libblas and liblapack names pointing to the appropriate mkl libraries. We would also have to create some symlinks to the have {prefix}/lib directory point to MKL's lib directory. The Intel package is a big mess, eh?!

Such a mess. But yeah, we should probably create symlinks for libblas/liblapack. Otherwise, every package that depends on blas/lapack will need a big if-statement for linking to various providers.

Plus, you can't have a package depend on blas or mkl. Providing blas/lapack/fft gives us the maximum flexibility.

this will be trickier than just providing a symlink, as other MKL libraries will need to be lined in (refer to https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor). There are additional complexities w.r.t compiler vendor, sequential vs openmp, etc. Suffice it to say, I don't think we can easily just provide blas and lapack.

Great. I wonder if it's possible to expand depends_on to allow something like depends_on('blas', 'mkl') so it can depend on one or the other. Would this solve the problem?

adamjstewart · 2016-04-19T17:32:47Z

var/spack/repos/builtin/packages/intel/package.py

+        install_script('--silent', 'install.ini')
+
+        absbindir = os.path.split(os.path.realpath(os.path.join(self.prefix.bin, "icc")))[0]
+        abslibdir = os.path.split(os.path.realpath(os.path.join(self.prefix.lib, "intel64", "libimf.a")))[0]


Why use os.path.split and os.path.realpath? I guess it doesn't matter too much since they will be replaced once #558 is merged.

The license file must exist in the absolute path of compiler binaries. Since prefix.bin is an actual directory and not a symlink, I use realpath. The split is to get the directory path of the icc binary.

lee218llnl · 2016-04-22T13:59:32Z

the latest commit uses searches a pset file for possible components and then uses a regex to specify features to install. This should be more portable across versions than manually specifying all components.

adamjstewart · 2016-04-22T14:10:25Z

var/spack/repos/builtin/packages/intel/package.py

+
+        components_string = ''
+        for component in components:
+            components_string += component + ';'


components_string = ';'.join(components)

adamjstewart · 2016-04-22T16:54:23Z

I know we discussed this in #761, but are there still plans to make separate intel (compilers only), mkl, daal, etc packages? If so, we might want to rename this package intel-suite.

davydden · 2016-04-26T04:00:03Z

var/spack/repos/builtin/packages/mkl/package.py

+    version('11.3.2.181', '536dbd82896d6facc16de8f961d17d65',
+            url="file://%s/l_mkl_11.3.2.181.tgz" % os.getcwd())
+
+    #provides('mkl')


shouldn't this be provides('blas') + provides('lapack') ? together with something like

def setup_dependent_package(self, module, dspec): libdir = find_library_path('libmkl_rt.%s' % dso_suffix, self.prefix.lib64, self.prefix.lib) # TODO: what to do with static libs? # self.spec.blas_static_lib = join_path(libdir, 'libopenblas.a') # self.spec.lapack_static_lib = self.spec.blas_static_lib # For now to make life easier stick with run-time MKL library. # Otherwise we would need the whole list of libraries... :-( if '+shared' in self.spec: self.spec.blas_shared_lib = join_path(libdir, 'libmkl_rt.%s' % dso_suffix) self.spec.lapack_shared_lib = self.spec.blas_shared_lib

lee218llnl · 2016-04-29T17:30:36Z

I moved the IntelInstaller helper base class to packages/intel.py. The other packages, like MKL can get to it via:
from spack.pkg.builtin.intel import IntelInstaller
Thanks to @tgamblin for the pointer on how to do this. Let me know if this seems like a reasonable setup.

Add hydra package

Added Adol-C package

Prevent use of system GTK+

- also reworded the boilerplate a bit.

lee218llnl · 2016-05-12T21:32:16Z

lib/spack/spack/package.py

            spack.repo.get(self.extendee_spec)._check_extendable()

+    @property
+    def global_license_dir(self):


@adamjstewart @tgamblin I added this new global_license_dir property so the Intel base package can overload the global_license_file property and have some assurances it's using the correct directory by referencing global_license_dir.

lee218llnl · 2016-05-12T21:33:46Z

@tgamblin I hope rebasing develop to pick up #558 was OK.

adamjstewart · 2016-05-12T21:37:14Z

Haha, it looks like you messed up your rebase somehow. You definitely didn't affect 644 files.

lee218llnl · 2016-05-12T22:10:48Z

@adamjstewart yeah, clearly my GIT-fu is not very good... any guidance on how to fix this?

davydden · 2016-05-12T22:29:50Z

i would say first of all have a back-up git checkout -b features/intel_backup. Then go back to your current branch git checkout features/intel.

(1)
It looks like you have a mixture of your commits (like the first one) and everything else that happened on develop branch. My guess is that if you go to develop, pull, then go back to your branch and do git rebase develop, then all the repetitions should go away and you will be left only with your stuff you are adding in this PR.

(2)
Another way is to merge and reset, but loose the history (effectively stash things or re-create commits). Along the lines of

# starting from your current messed-up branch
git remote update # update remotes
git merge origin/develop  # merge the current develop to your branch to make sure they are in sync

git checkout -b temporary # go to temp branch
git reset origin/develop # reset all commits to the HEAD of develop, that does NOT delete changes as this is a soft reset.

# at this point you will have un-commited changes
# so you can examine them, add changed files and commit as usual
git status
git diff
git add
git commit 

# if all is good, go back to the messed up branch
git checkout features/intel
# hard reset it to the one you cleaned up.
git reset --hard temporary 
# now repush
git push [...] --force

But don't rush it out, i would wait until someone else suggest a way to clean it up or comment on what i suggest.

…r compilers

…eatures/intel

lee218llnl · 2016-05-12T23:18:09Z

wow, I don't think I helped my cause any more... There are only a few files for me to add/modify, so I may just close this PR without merging and open a new one.

lee218llnl · 2016-05-13T03:20:20Z

Due to my ugly rebase attempt and inability to fix, this is being closed in favour of #946.

…ck#796)

adamjstewart reviewed Apr 19, 2016
View reviewed changes

lee218llnl mentioned this pull request Apr 19, 2016

Packaging of Intel Compilers #761

Closed

adamjstewart reviewed Apr 19, 2016
View reviewed changes

adamjstewart reviewed Apr 22, 2016
View reviewed changes

lee218llnl mentioned this pull request Apr 25, 2016

Packages with shared setup infrastructure #832

Closed

davydden reviewed Apr 26, 2016
View reviewed changes

bodduv and others added 14 commits May 2, 2016 14:31

missing dependencies (m4) added for netcdf and autoconf

e111add

Add hydra package

b99c8b6

Remove OpenMPI dependency on hwloc

c110865

Add argcomplete python package

4e2154e

Merge pull request #810 from adamjstewart/features/hydra

6dad59c

Add hydra package

Add latest jpeg version

9c3d8da

Merge branch 'develop' into openmpi_variants

cfc25d0

Prevent use of system GTK+

61e5ee5

Un-fix version 3

17fa1b5

Merge branch 'develop' into features/argcomplete

7255f69

Merge pull request #875 from jppelteret/features/adol-c

568800c

Added Adol-C package

Merge pull request #708 from adamjstewart/fixes/qt

999dda8

Prevent use of system GTK+

dealii: add new dependency -- gsl

cde1d18

dealii: fix a bug where P4EST_DIR was unconditionally requested

2eb4248

tgamblin and others added 4 commits May 12, 2016 10:30

Fix #941: add copyright to new package files.

5e13bba

- also reworded the boilerplate a bit.

added license support to Intel packages from PR 558

cdb5494

added license support to Intel packages from PR 558

e1c2ac7

removed conflict tag

700c24c

lee218llnl reviewed May 12, 2016
View reviewed changes

lee218llnl mentioned this pull request May 12, 2016

Add ilp64 variant of OpenBLAS #280

Closed

Gregory L. Lee and others added 14 commits May 12, 2016 15:47

initial package

388f171

fixes suggested by adamjstewart

f8bbece

use regexs to list components

080b571

renamed intel to intel-parallelstudio

90c334a

added separate packages for ipp mkl and daal

fa17a03

moved shared install code to support/intel and added intel package fo…

30b25a6

…r compilers

moved IntelInstaller to intel/package.py

48a052e

added latest releases

e92bfe4

initial package

84d4671

added separate packages for ipp mkl and daal

c3c27cb

moved shared install code to support/intel and added intel package fo…

cabaa2f

…r compilers

moved IntelInstaller to intel/package.py

c994172

added license support to Intel packages from PR 558

18d8e1a

Merge branch 'features/intel' of https://github.com/LLNL/spack into f…

e5678ee

…eatures/intel

lee218llnl mentioned this pull request May 13, 2016

Intel software packages and license enhancements #946

Merged

lee218llnl closed this May 13, 2016

tgamblin deleted the features/intel branch August 29, 2016 16:58

olupton pushed a commit to olupton/spack that referenced this pull request Feb 7, 2022

ispc: new download url after sourceforge stop upload of releases (spa…

4a59d18

…ck#796)

Conversation

lee218llnl commented Apr 18, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lee218llnl commented Apr 22, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Apr 22, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lee218llnl commented Apr 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lee218llnl commented May 12, 2016

Uh oh!

adamjstewart commented May 12, 2016

Uh oh!

lee218llnl commented May 12, 2016

Uh oh!

davydden commented May 12, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lee218llnl commented May 12, 2016

Uh oh!

lee218llnl commented May 13, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants

davydden commented May 12, 2016 •

edited

Loading