Skip to content

move sbang to unpadded install tree root#19640

Merged
tgamblin merged 23 commits intodevelopfrom
bugfix/padding-and-sbang
Nov 13, 2020
Merged

move sbang to unpadded install tree root#19640
tgamblin merged 23 commits intodevelopfrom
bugfix/padding-and-sbang

Conversation

@becker33
Copy link
Copy Markdown
Member

@becker33 becker33 commented Oct 31, 2020

Fixes #19629.

Also changes how padding is specified.

Since #11598 sbang has been installed in the install_tree. This doesn't play nicely with install_tree padding, since sbang can't do its job if it's installed in a long path.

This PR changes the padding specification so padding can only be at the end of the string, and stores the install_tree root both with and without padding. The root without padding is stored as spack.store.store.short_root.

Now, sbang is installed to spack.store.store.short_root/bin. This path should remain short enough for sbang to work.

@becker33
Copy link
Copy Markdown
Member Author

@scottwittenburg @tgamblin @eugeneswalker This is the fix we discussed earlier today. Took some extra time to refactor for better separation of concerns between the store and backwards compatibility for configs (and get good test coverage).

@eugeneswalker
Copy link
Copy Markdown
Contributor

eugeneswalker commented Nov 3, 2020

config:
  install_tree:
    root: /spack-install
    padded_length: 40
$> spack install automake
...
==> Installing automake
==> Fetching https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/_source-cache/archive/b2/b2f361094b410b4acbf4efba7337bdb786335ca09eb2518635a09fb7319ca5c1.tar.gz
################################################################################################################################################################ 100.0%
==> automake: Executing phase: 'autoreconf'
==> automake: Executing phase: 'configure'
==> automake: Executing phase: 'build'
==> automake: Executing phase: 'install'
[+] /spack-install/spack_path_placeholder/sp/linux-ubuntu18.04-x86_64/gcc-7.5.0/automake-1.16.2-jw3uludcx2nyhmpxig5yu3hl52rsckxg

So here the padded install tree = /spack-install/spack_path_placeholder/sp/

With this PR, do we expect sbang to be at /spack-install/bin/sbang @becker33 ?

@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Nov 3, 2020

@eugeneswalker yes, that's correct.

Copy link
Copy Markdown
Contributor

@eugeneswalker eugeneswalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this a few different ways and couldn't find any misbehavior!

@scheibelp scheibelp self-assigned this Nov 3, 2020
Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few requests and questions

padding to length

Assumes path does not have a trailing path separator"""
padding_length = length - len(path) - 1 # -1 for separator
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(question) This logic handling the possibility of the padding length being only 1 larger seems new. Should it raise an exception? I assume if someone asks for padding, they need it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason it's new is because previously, the padding didn't have to start a new directory name, so we could patch foo/ to foos/. Now that we're storing the sbang at the top level in foo/bin/sbang, there is no way to pad foo to 4 characters. It's just not possible.

I'd be fine with having it raise an exception, but my guess is that it will A) never come up, and B) not be useful to users. I don't know of any use case for padding that actually depend on the exact number.

@becker33
Copy link
Copy Markdown
Member Author

@scheibelp I've addressed all concerns except the last one, and that one may take more conversation.

@becker33 becker33 force-pushed the bugfix/padding-and-sbang branch from 0cb2b5e to 532be88 Compare November 11, 2020 18:05
Copy link
Copy Markdown
Member

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have two minor requests but this LGTM

@carsonwoods FYI this PR will generate a minor conflict with #11871 (like #19209, I don't think there will be a conceptual conflict)



def parse_install_tree():
"""Parse old and new formats and return relevant values.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This docstring needs work. "relevant values" does not tell me anything about what it returns, as a naive reader has no idea what is relevant. This also doesn't tell me anything about the formats.

I'd expect there to be examples and descriptions of the formats here in the docstring, as well as description of the return type and what is in it. Use the google docstring format we use elsewhere, and add Arguments: and Returns: sections.

For testability and clarity, this should just take a dict as a parameter (I guess it should expect the result of config.get("config")) instead of calling config.get(). _store is the store-specific logic - let it call config.get and pass the result to parse_install_tree. This thing should just process a dict.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tgamblin addressed

@tgamblin tgamblin changed the title move sbang to pre-padding install tree move sbang to unpadded install tree root Nov 12, 2020
@tgamblin tgamblin merged commit fafff0c into develop Nov 13, 2020
@tgamblin tgamblin deleted the bugfix/padding-and-sbang branch November 13, 2020 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

autoconf build partially non-functional when install_tree is padded (sbang ??)

5 participants