Skip to content

CentOS and RHEL report as the same thing. #6373

Closed
briedel wants to merge 36 commits intospack:developfrom
briedel:develop
Closed

CentOS and RHEL report as the same thing. #6373
briedel wants to merge 36 commits intospack:developfrom
briedel:develop

Conversation

@briedel
Copy link
Copy Markdown
Contributor

@briedel briedel commented Nov 20, 2017

No description provided.

@adamjstewart
Copy link
Copy Markdown
Member

Not saying I'm opposed to this PR, but the distro.py file is not the right place to make this change. If a new release of distro comes out, we will likely blow away this file with the new one. And if we ever release on PyPI, we may drop many of these external modules altogether. I'm not sure where the right place to do this would be, but it would not be here.

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Nov 20, 2017

Is there another place where the distro is defined or can be overwritten somehow? The compiler.yaml can be used to overwrite it, but I have had issues with that trick.

@adamjstewart
Copy link
Copy Markdown
Member

I would override it in lib/spack/spack/operating_systems/linux_distro.py. That's the file where we actually call external.distro. There are already hacks in there for Ubuntu.


homepage = "http://toolkit.globus.org"
url = "http://toolkit.globus.org/ftppub/gt6/installers/src/globus_toolkit-6.0.1470089956.tar.gz"
url = "http://toolkit.globus.org/ftppub/gt6/installers/src/globus_toolkit-6.0.1493989444.tar.gz"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a separate PR.

version('4.3.0', '39c2fab9f632f35e12ff607ccaf9e16c')

depends_on('[email protected]:', type='build')
depends_on('zlib')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you submit a PR, it's best practice to submit it on a branch, like features/centos-rhel, not on develop. Your PR can only contain a single change, whereas you may want to merge multiple changes into your develop branch. What I'm saying is, either redo this PR on a branch, or freeze the branch until this PR is merged (which may take a while).

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Dec 11, 2017

Are there any open questions about this PR?

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Jan 15, 2018

What are the open issues with this?

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 1, 2018

@tgamblin Could this be merged?

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Feb 3, 2018

Is there any issue where the rationale for this PR is explained?

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 4, 2018

There isn't an open issue. I can create one if need be.

The main issue is that CentOS, RHEL, and SL should be reported as the same OS, but CentOS is reported differently than RHEL/SL.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Feb 4, 2018

The main issue is that CentOS, RHEL, and SL should be reported as the same OS, but CentOS is reported differently than RHEL/SL

I get that this is what the PR does, my question is more "why CentOS needs to be reported the same as rhel"?

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 4, 2018

When distributing software across OSG, we will end up on machines that are either SL or CentOS machines. They need to be reported the same for us to be able to use Spack's internal OS resolution.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Feb 4, 2018

Sorry, I still don't understand why you can't distribute software taking into account where the request comes from (software built for rhel to rhel machines, same for centos)?

If I look at the PR with my current understanding of things, this seems to me more a regression than a feature - Spack erroneously reporting centos as rhel.

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 4, 2018

It means that instead of two machines (SL/CentOS6 and SL/CentOS7), I need to operate 4 machines (SL6, CentOS6 SL7, and CentOS7) and build everything twice to distribute software. Are there any packages which rely on the difference between CentOS and SL/EL? Is there a way this feature could be turned on and off somehow rather than having to hardcoded?

@adamjstewart
Copy link
Copy Markdown
Member

Based on #1075 (comment) @gartung may also be interested in this PR.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Feb 4, 2018

@adamjstewart Thanks for the link. Actually I agree a lot more with what @tgamblin says:

I think it's useful to know that something was built on CentOS, in case that actually matters later for debugging. But that doesn't mean we couldn't allow Spack to have looser semantics on install. i.e., the user could allow using a CentOS-built binary if a RHEL one isn't available, or Spack could just know that these two are generally compatible. That would get you binary caching but it would also avoid losing information from the build.

than with this PR. @briedel If the problem is operating physical machines, we can work on constructing a binary cache using dockerized builds. I think that + the feature proposed in the quote above are better than reporting the wrong OS.

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 4, 2018

It is not about operating physical machines. It is about not having to compile and manage ~100 packages that we want to distribute. I would prefer if there were a knob like would allow me to say "I don't care about the difference in EL distros. They are all the same to me"

@gartung
Copy link
Copy Markdown
Member

gartung commented Feb 4, 2018

I understand why @briedel did this. Scientific Linux CERN 6 reports as RHEL6. CERN Centos7 reports as Centos7. For HEP purposes, they should both report as RHEL. This can be a patch that HEP applies to spack usage since it is HEP specific.

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 4, 2018

Let me rephrase the question. Where can I put in the knob that allows me to turn this on or off?

@gartung
Copy link
Copy Markdown
Member

gartung commented Feb 5, 2018

HEP operates a heterogenous grid of compute nodes. The only requirement imposed is binary compatibility with RHEL. There are many RHEL clones besides Centos and Scientific Linux. It would be preferable to build a package once for any RHEL6 clone.

Is it possible to use symbolic links in opt/spack to allow listing for common RHEL clones?

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Feb 23, 2018

@alalazo Preferred solution is just to create a symlink?

@gartung
Copy link
Copy Markdown
Member

gartung commented Jun 24, 2019

@briedel this has aged long enough. We just applied this patch to our fork:
FNALssi#7

@briedel
Copy link
Copy Markdown
Contributor Author

briedel commented Jul 2, 2019

Sounds good @gartung Thanks.

@briedel briedel closed this Jul 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants