Skip to content

buildcache push always --allow-root#38878

Merged
alalazo merged 7 commits intospack:developfrom
haampie:buildcache/make-allow-root-a-noop
Jul 18, 2023
Merged

buildcache push always --allow-root#38878
alalazo merged 7 commits intospack:developfrom
haampie:buildcache/make-allow-root-a-noop

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented Jul 13, 2023

closes #7237
closes #11620

Without --allow-root spack cannot push binaries that contain paths in
binaries. This flag is almost always needed, so I don't see the point of
requiring users to spell it out. Even without --allow-root, rpaths would
still have to be patched, so the flag is not there to guarantee binaries
are not modified on install, and in fact relocation is still in dangerous
territory, because patchelf will have to rewrite the dynamic section,
which hasn't been without issues.

So, this PR makes --allow-root the default, and drops the code required
for it.

This also allows us to drop binutils strings as a dependency.

@spackbot-app spackbot-app bot added binary-packages commands core PR affects Spack core functionality tests General test capability(ies) labels Jul 13, 2023
haampie added 3 commits July 17, 2023 11:07
Without --allow-root spack cannot push binaries that contain paths in
binaries. This flag is almost always needed, so I don't see the point of
requiring users to spell it out. Even without --allow-root, rpaths would
still have to be patched, so the flag is not there to guarantee binaries
are not modified on install.

So, this PR makes --allow-root the default, and drops the code required
for it.
@haampie haampie force-pushed the buildcache/make-allow-root-a-noop branch from 8c5615b to 52da80a Compare July 17, 2023 09:07
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM. Maybe remove something more.

print("Relocatable nodes")
print("--------------------------------")
print(spec.tree(status_fn=spack.relocate.is_relocatable))
print(spec.tree(status_fn=lambda s: True))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we can completely remove the:

spack buildcache preview command

It doesn't seem to be synced with other commands, and it's not env aware...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this was introduced in #9199 We can safely remove it if we get rid of --allow-root.

Copy link
Copy Markdown
Member Author

@haampie haampie Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I'll just remove the output of the command, but leave the deprecation message, so it doesn't error if someone had it in their ci script or so.

@alalazo alalazo self-assigned this Jul 17, 2023
alalazo
alalazo previously approved these changes Jul 17, 2023
@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 17, 2023

Hi @gartung, since you've worked on this before, can you share your opinions?

Two parts:

  1. --allow-root is almost always necessary, so it makes sense to use that as the default.

  2. The old behavior could be preserved by adding a flag --disallow-root (or something better), BUT, in my opinion it's not worth it, given that:
    a. There's almost always at least one package that is not relocatable (modulo rpath), making spack buildcache push error
    b. Spack uses (long => short prefix) relocation in its package CI, so we trust that it works
    c. I do not always trust patchelf to relocate from short => long prefix, it has caused many headaches in the past, so --disallow-root would only give a false sense of security

Dropping the old behavior also allows us to drop the dependency on strings, which improves usability.

@gartung
Copy link
Copy Markdown
Member

gartung commented Jul 18, 2023

I am OK with this as long as it passes regression tests. It it meant to catch compiled in paths that some older programs use to locate config files, etc.

@gartung gartung self-requested a review July 18, 2023 13:58
gartung
gartung previously approved these changes Jul 18, 2023
…tion message -- dont remove it entirely, since that may pbreak ci for people who may use this command (if any)
@haampie haampie dismissed stale reviews from gartung and alalazo via 5024a57 July 18, 2023 14:41
@alalazo alalazo merged commit 5b23c5d into spack:develop Jul 18, 2023
@haampie haampie deleted the buildcache/make-allow-root-a-noop branch July 18, 2023 17:14
eugeneswalker pushed a commit to eugeneswalker/spack that referenced this pull request Jul 22, 2023
…on (spack#38878)

Without --allow-root spack cannot push binaries that contain paths in
binaries. This flag is almost always needed, so there is no point of
requiring users to spell it out. 

Even without --allow-root, rpaths would still have to be patched, so the 
flag is not there to guarantee binaries are not modified on install.

This commit makes --allow-root the default, and drops the code 
required for it. It also deprecates `spack buildcache preview`, since 
the command made sense only with --allow-root.

As a side effect, Spack no longer depends on binutils for relocation
mpokorny pushed a commit to mpokorny/spack that referenced this pull request Sep 18, 2023
…on (spack#38878)

Without --allow-root spack cannot push binaries that contain paths in
binaries. This flag is almost always needed, so there is no point of
requiring users to spell it out. 

Even without --allow-root, rpaths would still have to be patched, so the 
flag is not there to guarantee binaries are not modified on install.

This commit makes --allow-root the default, and drops the code 
required for it. It also deprecates `spack buildcache preview`, since 
the command made sense only with --allow-root.

As a side effect, Spack no longer depends on binutils for relocation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binary-packages commands core PR affects Spack core functionality dependencies tests General test capability(ies) update-package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Un-relocatable package in DAG prevents binary cache creation for other packages in DAG Mark a package as non-relocatable

3 participants