@amundson I'm trying to start thinking through the corner cases of this.

I think this will need some special handling for extendable packages. Extendable packages we need to be able to write to their prefix to activate extensions, and we can't guarantee that a spack instance can write to locations in an instance with which it is chained.

The obvious answer is symlinks from the extendable package to its prefix in the local spack instance, but I don't think that will work. Packages like python get their location at compile time and need the extensions to be there.

My current thought is that we will need to add some sort of local flag to database queries, and when local=False only include non-extendable packages in the query. Interestingly enough, I think this won't be a problem for extensions, even if they are non-local and extend a local packages.

@amundson Can you please look at #642 and see if there's anything additional to be gleaned from it?

The Spack Server idea is described at #3156. @becker33 suggests that Shared Spack, which we have today, does mostly the same thing.

Without having read it in detail, this seems to be similar to spack stack

(EDIT: which can be found at #7081)

Without having read it in detail, this seems to be similar to spack stack

@obreitwi @amundson
Now I'm confused... what's the difference between Spack Stack and Spack Chain?

Now I'm confused... what's the difference between Spack Stack and Spack Chain?
Judging from five minutes of investigation spack stack and spack chain seem to be two solutions to essentially the same problem:

• spack stack symlinks all currently installed packages from another spack installation into the current one, allowing packages to be built against pre-built packages from a system-wide repository. This is helpful if you want to test changes to spack in a local context or need extra packages on top of our rather large system-wide install.
• spack chain, from what I can tell, seems to achieve the same by actually submitting db.querys to the package databases in several spack repositories instead of symlinking. A package is first checked for in the local repository, then in the next entry in the chain and so on. This should allow newly-installed packages to be found automatically (you need to re-stack for that). I did not check if spack chain works in all cases (e.g., if some parts of the spack codebase still implicitly expect one global database).

It sounds like these two features are quite similar. And that Spack Chain can do some thing sthat Spack Stack cannot. @obreitwi Can you think of use cases that Stack Spack would be good for but not Spack Chain? Or is there a way to combine the functionality of the two features into one? I think we should try to avoid maintaining multiple features that do similar things in slightly different ways.

@obreitwi I think that spack chain and spack stack are different implementations of the same feature. I happen to prefer the implementation in this PR, for a couple of reasons. One, this will know when a package "upstream" is uninstalled, rather than leaving broken symlinks. Second, some packages do not behave well when symlinked.

@obreitwi If spack stack were removed and replaced with spack chain, would it work for you?

I haven't had much to add to this conversation because I see others recognizing the advantages of the chain approach. However, I think there is one advantage over the stack approach that has not yet been discussed. Can you tell which package came from which repository in Spack Stack? In my chain PR the spack find command has been modified to list the packages in each chained repository separately.

@obreitwi If spack stack were removed and replaced with spack chain, would it work for you?

Yes, absolutely. spack stack was just a quick and easy fix to built packages on top of another. spack chain seems much better integrated in the given spack workflow.

Plus: In the meantime our approach of stacking spack repositories has shifted from spack stack to building spack in singularity containers onto which each user can add his own overlay if he so desires.

Therefore dropping spack stack is absolutely fine!

@amundson, all: I really like this feature. I'll try to do a full review soon. I'd like to get this or something like it integrated.

@obreitwi:

Yes, absolutely. spack stack was just a quick and easy fix to built packages on top of another. spack chain seems much better integrated in the given spack workflow.

Awesome, and thanks for the stack PR, too!

@amundson FYI with flake8 on your path you can do spack flake8 locally to reproduce Travis' checks. To get flake8 either do a pip install or spack install py-flake8 && spack activate py-flake8

I would be interested in testing this PR, currently there are conflicts.
Would it be possible to rebase onto current dev?
Thanks a lot in advance

Ping @amundson @tgamblin @citibeth @alalazo Any hints on when this feature can be tested and which is best approach to overlay spack installations in multiuser environment
Thanks in advance

My assumption with this feature is that inside userlike, running spack install with a package that already exists inside systemlike should link the installation from there instead of building the package again. I configure systemlike following the above and install a package, then configure userlike to see the packages inside systemlike by adding to chain_prefixes inside etc/spack/defaults/config.yaml. Now running spack find inside userlike lists the package from systemlike as well. Then I run spack install inside userlike with a package that has a dependency on the package that exists already inside systemlike. I expect spack to display a different behavior than when installing from scratch and link the dependency from systemlike. However, spack will download and build the dependency again inside userlike.

Am I misunderstanding the usage of this feature? It should guard against having duplicate packages in different environments like systemlike and userlike, however I have not managed to replicate this behavior. I have used the latest version of this branch and I have also tried to checkout the version from May the 3rd with no success. I am testing using pkgconfig and ncurses, the latter depends on the former.

I expect spack to display a different behavior than when installing from scratch and link the dependency from systemlike. However, spack will download and build the dependency again inside userlike.

@paul-chelarescu my understanding from reading through the PR is that it intends to support this. It may be useful to include the paths for all the Spack installations you are testing with, and the contents of your configuration files if possible.

@amundson @gartung I've started looking at this, would you want to chat about it during the Thursday telcon on 6/28?

@paul-chelarescu Try out #8545

@gartung I am having the same result with that PR as well.

I see the problem. The system-like db is not being linked into the user-like db. When I try linking them I end up with infinite recursions that I am trying to debug.

@scheibelp @tgamblin This PR is broken since commit f202198 was merged. Since then I am unable to find a way to add the parent database without infinite recursion.

I wanted to make sure: should we continue conversation in #8545 or here?

We should move the discussion to #8545
I am updating this for Jim while he is away on vacation so I can't merge my branch into his.

@paul-chelarescu Can you try #8545 again

@gartung I have and I am still getting the same behavior. Spack does not use the dependencies inside systemlike and instead is building them again. I have added both systemlike and userlike paths to chain_prefixes.

The spack chain configuration has been reworked in
#8772
so I am closing this pull request.