New packages rdma-core and libnl. by eschnett · Pull Request #4168 · spack/spack

eschnett · 2017-05-09T15:14:31Z

Add “rdma” variant to openmpi that uses the Spack-installed rdma-core library for ibverbs.

adamjstewart · 2017-05-09T15:19:09Z

var/spack/repos/builtin/packages/libnl/package.py

+
+    def configure_args(self):
+        if not sys.platform.startswith('linux'):
+            raise InstallError("libnl requires Linux")


@alalazo @tgamblin Is there a way to put this in conflicts? I know platform=darwin works but I'm not sure about platform=linux.

I think the point here is the absence of logical operators in constraints so far. You can say in whatever directive:

conflicts('platform=linux')

but you can't say:

conflicts('! platform=linux')

or

conflicts('! (platform=linux | platform=darwin)')

Good point. This is fine for now then. We can revisit this if the concretizer ever gets those features.

adamjstewart · 2017-05-09T15:58:41Z

Flake8:

var/spack/repos/builtin/packages/openmpi/package.py:200: [E713] test for membership should be 'not in'
var/spack/repos/builtin/packages/openmpi/package.py:201: [E501] line too long (82 > 79 characters)
var/spack/repos/builtin/packages/rdma-core/package.py:41: [F841] local variable 'spec' is assigned to but never used

eschnett · 2017-05-10T13:43:22Z

Corrected Flake8 failures.

adamjstewart · 2017-05-10T14:34:47Z

var/spack/repos/builtin/packages/libnl/package.py

+    version('3.3.0', 'ab3ef137cad95bdda5ff0ffa5175dfa5')
+    version('3.2.25', '03f74d0cd5037cadc8cdfa313bbd195c')
+
+    depends_on('bison', type='build')


#4188 also adds a libnl package. Are you missing some dependencies here?

Yes I did. I added them.

svenevs · 2017-05-11T23:09:51Z

var/spack/repos/builtin/packages/libnl/package.py

+    """libnl - Netlink Protocol Library Suite"""
+
+    homepage = "https://www.infradead.org/~tgr/libnl/"
+    url      = "https://github.com/thom311/libnl/releases/download/libnl3_3_0/libnl-3.3.0.tar.gz"


So officially according to the website 3.2.25 is the latest stable. I actually had the same urls as you from github to start, but @adamjstewart has a strong preference for non-github tags (since they can be changed).

It's generally unclear to me what the state of this project is...

As packaged in Fedora:

dnf info libnl3 Last metadata expiration check: 5 days, 7:42:11 ago on Sat May 6 11:25:52 2017. Installed Packages Name : libnl3 Arch : x86_64 Epoch : 0 Version : 3.2.27 Release : 1.fc23 Size : 819 k Repo : @System From repo : updates Summary : Convenience library for kernel netlink sockets URL : http://www.infradead.org/~tgr/libnl/ License : LGPLv2 Description : This package contains a convenience library to simplify : using the Linux kernel's netlink sockets interface for : network manipulation

Who is the maintainer? tgraf or thom311?

Edit: to be clear, it seems to me that thom311 is the maintainer now? It's just unclear what the official project is, the website linked is from tgraf (same website fedora lists too, and the google).

It looked to me as if the web page was outdated, and the github repo was more authoritative.

Yes, github tags can be changed, but that's why we have the md5 checksums, no? After all, tarballs can also be replaced.

@adamjstewart has a strong preference for non-github tags (since they can be changed).

To clarify, I have a strong preference for the official download site. For Python packages, this is usually PyPI, but for other packages it's whatever the developer primarily distributes their software from.

Yeah, I ordinarily agree but I think this case is the exception. The GitHub url's that @eschnett has are likely the ones we want, the 3.2.25 "latest stable" on the "official" site is very old, and the GitHub fork from thom311 is ~300 commits ahead. I'm asking because I want to make sure mine doesn't accidentally overwrite things. TBH this one is going to merge before pcl, should I just delete my libnl?

@svenevs Yeah, if this PR is merged first you can just delete yours or update anything else you think it is missing.

adamjstewart · 2017-05-12T15:39:04Z

var/spack/repos/builtin/packages/libnl/package.py

+            raise InstallError("libnl requires Linux")
+
+        args = []
+        return args


You can actually do this without overriding configure_args. It would look like:

@run_before('autoreconf') def check_platform(self): if not sys.platform.startswith('linux'): raise InstallError('libnl requires Linux')

It should build without issue on bsdlike systems as well. To be honest, the docs on this have always been vague to me, but I think the "linux trap all" is

if not (sys.platform.startswith('linux') or sys.platform.startswith('freebsd')): raise InstallError("libnl requires Linux")

alalazo · 2017-05-12T15:40:45Z

var/spack/repos/builtin/packages/openmpi/package.py

    patch('configure.patch', when="@1.10.0:1.10.1")
    patch('fix_multidef_pmi_class.patch', when="@2.0.0:2.0.1")

+    variant('rdma', default=False, description='Build with rdma-core; also need to manually set "fabrics" to include "verbs"')


Would it be worth to just add a new value to mv variant fabrics? In this way you'll avoid the check at run-time, and you won't add a new variant (i.e. you'll be backward compatible with respect to changing hashes).

Add “rdma” variant to openmpi that uses the Spack-installed rdma-core library for ibverbs.

…ad of a variant

shamisp · 2017-11-06T19:55:55Z

var/spack/repos/builtin/packages/openmpi/package.py

        default=None if _verbs_dir() is None else 'verbs',
-        description='List of fabrics that are enabled',
-        values=('psm', 'psm2', 'pmi', 'verbs', 'mxm'),
+        description=("List of fabrics that are enabled ('rdma' enables "


Verbs layer calls it "providers". Typically each interconnect vendor provides one for their hardware. Full least providers is here https://github.com/linux-rdma/rdma-core/tree/master/providers

shamisp · 2017-11-06T20:00:07Z

var/spack/repos/builtin/packages/openmpi/package.py

+        description=("List of fabrics that are enabled ('rdma' enables "
+                     "'verbs', but implemented via the 'rdma-core' Spack "
+                     "package)"),
+        values=('psm', 'psm2', 'pmi', 'rdma', 'verbs', 'mxm'),


This is a bit confusing (but maybe I don't read it right) - all these (with the exception of verbs) are separate APIs that are not part of rdma-core. Some of these don't use rdma-core at all.

adamjstewart · 2018-04-25T15:33:49Z

Closed by #7833 and #7835. Reopen if you think there's anything missing.

adamjstewart reviewed May 9, 2017

View reviewed changes

adamjstewart mentioned this pull request May 10, 2017

[WIP] Add pcl package and friends, first attempt. #4188

Closed

15 tasks

adamjstewart reviewed May 10, 2017

View reviewed changes

svenevs reviewed May 11, 2017

View reviewed changes

adamjstewart reviewed May 12, 2017

View reviewed changes

alalazo reviewed May 12, 2017

View reviewed changes

eschnett added 8 commits May 19, 2017 14:51

New packages rdma-core and libnl.

6bf678b

Add “rdma” variant to openmpi that uses the Spack-installed rdma-core library for ibverbs.

rdma-core: Remove unused variable

7cada9f

openmpi: Reduce line length

da4cdbf

libnl: Add missing dependencies

f6feca3

libnl: Check platform in separate function

0762e39

rdma-core: Check platform in separate function

c5d9a72

openmpi: Implement ‘rdma-core’ dependency via additional fabric inste…

1a0cddf

…ad of a variant

rdma-core: Correct platform check

c700d3e

eschnett force-pushed the eschnett/rdma-core branch from 17b6f35 to c700d3e Compare May 19, 2017 18:51

shamisp reviewed Nov 6, 2017

View reviewed changes

baberlevi mentioned this pull request Jan 17, 2018

add better support for libfabrics #6978

Closed

This was referenced Apr 19, 2018

libnl: new package #7833

Merged

rdma-core: new package #7835

Merged

adamjstewart closed this Apr 25, 2018

Conversation

eschnett commented May 9, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alalazo May 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented May 9, 2017

Uh oh!

eschnett commented May 10, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

svenevs May 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Apr 25, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

alalazo May 9, 2017 •

edited

Loading

svenevs May 11, 2017 •

edited

Loading