Skip to content

[WIP] Enhancement Proposal: Mirror Improvements #705

Closed
xjrc wants to merge 5 commits intospack:developfrom
xjrc:features/mirror-improvements
Closed

[WIP] Enhancement Proposal: Mirror Improvements #705
xjrc wants to merge 5 commits intospack:developfrom
xjrc:features/mirror-improvements

Conversation

@xjrc
Copy link
Copy Markdown
Member

@xjrc xjrc commented Mar 31, 2016

The changes in this pull request aim to improve support for mirrors to make it more straightforward to support installation strategies for proprietary/licensed packages (see #553 and this discussion) and to make it easier to test/develop infrastructure related to mirror/local package fetching.

For package developers, the following changes in this PR are the most relevant:

  • Addition of a mirror_only flag to the package version function, which indicates that the given version of the given package can only be retrieved through local mirrors.
  • Support for creating mirror_only default packages, which is useful when creating Spack installation scripts for primarily proprietary packages (e.g. the PGI compiler; see Add Licensed Software Support #558).
  • Support for providing installation instructions to users for mirror_only versions of packages if these packages cannot be found in Spack mirrors.

While this pull request is still being actively developed, the following section will enumerate all of the work that still needs to be done:

  • Add support for the mirror_only flag to the version package function.
  • Implement the provision of installation instructions for mirror_only packages that cannot be found in local mirrors.
  • Implement mirror_only by default packages.
  • Separate the mirror fetching functionality into its own FetchStrategy subclass.
  • Implement automatic path detection for package tarballs contained in mirrors.
  • Remove the mirror_only flags from the Package.[do_patch|do_stage|do_fetch] functions and change the test cases that use these flags to use mirror_only package versions instead.

@citibeth
Copy link
Copy Markdown
Member

citibeth commented Apr 1, 2016

Don't forget to update the documentation before the PR is merged!

@tgamblin
Copy link
Copy Markdown
Member

@xjrc: have you seen #558? Does that PR cover your use cases here?

@xjrc
Copy link
Copy Markdown
Member Author

xjrc commented May 10, 2016

@tgamblin: I've seen PR #558 and I don't believe that it covers the functionality that I'd like to implement here. In particular, #558 (and please @adamjstewart correct me if I'm wrong here) aims to make it easier to set up and specify licenses for licensed software while this PR aims to simplify the process of specifying and using archives for packages that aren't publicly available. I imagine, though, that the features delivered by these PRs can be used together to improve a number of packages (since many packages that are licensed are also mirror only).

@citibeth
Copy link
Copy Markdown
Member

I installed something that wasn't publicly available without any of these enhancements:

  1. Create a new mirror directory
  2. Manually download the tarball and put it in the new mirror directory

The only glitch is that now Spack prints a lot of spurious "can't find" messages. But it basically works.

@xjrc
Copy link
Copy Markdown
Member Author

xjrc commented May 10, 2016

@citibeth: That's the workflow that I've been using for all of my packages with private archives as well. The goal with this PR is to remove a lot of the cruft in this workflow (e.g. the "can't find" messages, the need for a url field in packages that don't need it, etc.) and to make it clearer when an archive must be installed in a local mirror to be used in package installation.

@tgamblin tgamblin added the WIP label Sep 21, 2016
@alalazo
Copy link
Copy Markdown
Member

alalazo commented Apr 19, 2017

@xjrc Is this PR still actively developed?

@xjrc
Copy link
Copy Markdown
Member Author

xjrc commented Apr 19, 2017

@alalazo: No, I haven't worked on this PR in quite some time. I've only kept it open because I've considered getting back to it, but no luck thus far.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Nov 11, 2019

Closed as abandoned. Feel free to reopen if you restart working on this.

@alalazo alalazo closed this Nov 11, 2019
matz-e pushed a commit to matz-e/spack that referenced this pull request Apr 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants