@adamjstewart @tgamblin @scheibelp @davydden @alalazo

Summary

Virtual packages are a nice idea, but they have produced many problems as currently implemented. They make concretization slower and more complex, and they also seem to be at the root of a number of concretization bugs that seemingly never get fixed; see #7953.

I propose that we solve these problems by getting rid of virtual packages as a special case; and instead, implementing their functionality using general-purpose Spack machinery. This would result in a simpler and more robust system, while solving a bunch of nasty bugs all in one go.

Details

Create a new subclass of Package, which we will call VirtualPackage. Implementations will look like this:

class Mpi(VirtualPackage):
    variant('provider', default='openmpi', values=('openmpi', 'closedmpi'), multi=False)

    version(')
    version(')
    version(')

    depends_on('openmpi@1., when='provider=openmpi and @)
    depends_on('[email protected]:1., when='provider=openmpi and @)
    depends_on('[email protected]:1.7.5:', when='provider=openmpi and @)
    depends_on('closedmpi', when='provider=closedmpi')

    def setup_dependent_environment(self, spack_env, run_env, dependent_spec):
        return self.spec[self.spec.provider].setup_dependent_environment(spack_env,run_env, dependent_spec)

Some of this boilerplate could be wrapped up into VirtualPackage.

Pros

1. Gets rid of a lot of buggy complexity in core Spack.
2. It's something we can build today while eliminating long-standing bugs that cuase real problems.
3. Collects all information about each virtual package into one place. Currently it's spread around:
   1. Default provider is buried in a config file somewhere.
   2. Relation between virtual dep version and provider version is distributed among the providers (and buggy).
4. Makes it easy for Spack users to research what providers are available for a particular virtual package.

Cons

1. Introduces a need for a top-level file for each virtual dep. Providers can no longer "magically" provide a virtual dep; they need to register with the virtual dep's package to indicate that relationship.

DIscussion

I believe the pros outweigh the cons, if just to get rid of the bugs. In any case, any inconvenience introduced by requiring registration would be minor. There are only a handful of virtual packages, with a fairly static set of providers. Reuquiring virtual dep providers to register is not a significant burden. It might even be easier than what we have now; where virtual package info is spread around a few different places.

Questions

Is is currently possible to have when clauses like when='provider=openmpi and @?