I tried to use versioned virtuals, but I ran into a problem. The problem is that a package and the provider cannot share the same name, e.g. the FFTW package cannot contain a statement like providers('fftw').

When you try it, you get a complaint from Spack.

So, I think the options were:

1. Create FFTW2 and FFTW3 providers, since there API are not backwards compatible any way and there are not minor API revisions in-between, e.g. FFTW2.5
2. Rename the FFTW package to something else and reclaim FFTW name as a provider class. I think that kind of change would affect a lot of packages because it could break a lot of dependencies.

In principle, I suppose either would work but change 2. is more complicated IMO. But maybe that is the right thing to do.

(FYI, I am not aware of an FFTW@1 interface existing while in my academic lifetime going all the way back into the late 1990's. As a practical matter, only FFTW3 really matters. There are still a few legacy interfaces to FFTW2.)

If I understand you correctly, you mean can:

1. Can the FFTW package be a provider for all FFTW APIs

@naromero77 I think there's also option number 3:

• Name the virtual provider fftw-api (or something similar), so that you could maintain backward compatibility AND you can version the API like @tgamblin proposes.

I like option number 3. If there are no other suggestions, I will go with that route in the next day or two.

Here are the changes:

• Used version virtual provider interface using @alalazo suggested name
• Set default virtual provider to be FFTW
• Assertion message in virtual package tests points to location of file containing default virtual packages.

This is a good observation and one that crossed my mind. I don't have a simple answer, but I do have more information.

I think there are a number of things that are (unfortunately) entangled:

• There are a number of different compile options that give you a particular precision (double, single, etc.) and parallel versus serial.
• There are technically other APIs inside of FFTW that are bundled together, most obvious ones are 1D, 2D, and 3D FFTs. They are bundle together as function of the precision chosen and serial vs. parallel at build-time.
• Vendor provided FFTs that provide the FFTW API do not cover all these combinations. Especially not long double and quad precision.

My guest-estimate is that 85-90% of the use cases of FFTW are this combination:

• serial
• double precision real, double precision complex, 1D, 2D, 3D API (these are all bundled together)

Again, just guessing, I think the packages that have made it into Spack are probably this category which is the default build option for FFTW.

There is some use of single precision and parallel versions in the remaining 10-15% of usage. For the other precisions, long double and quad double -- don't know of any science code that uses it, but the fact that exists indicates that someone has need of it.

I think its OK to implement all 8 APIs, but it would result in a lot of clutter as @alalazo indicated. I would propose the following naming scheme:
fftw-[dp,sp,qp,ldp]-[serial,mpi]-api

If we implement a subset, I would recommend that we at least do:
fftw-[dp,sp]-[serial,mpi]-api

This would cover 99.9% of the use cases. We could add quad precision and long double (ldp), but then only FFTW would be able to provide it.

However, ,y recommendation is that we just two providers:
fftw-serial-api
fftw-mpi-api

With the exception of Intel, the other vendors, e.g. IBM ESSL, AMD ACML, do not provide the mpi version of the FFTW API but do provide single and double precion.

I looked through the FFTW manual this morning. They are quite a few APIs available, some of them I have never heard of. I am casting my vote that we only need two virtual providers:
fftw-mpi-api
fftw-api (maybe leave out "serial" in the name?)

Aside @alalazo, I have not received many comments on this PR @wscullin @tgamblin @adamjstewart @ax3l

From what I read in this thread it looks to me that as long as we don't properly support variants on virtual dependencies, a highly variadic API like FFTW is hard to abstract as a spack virtual package.

On top of this, afaik there is currently no way for a downstream package (not config) to express it prefers a specific provider of a virtual dependency. So what will happen is that people testing and developing (aka preferring) some specific fftw implementation will come here and complain ;-)

@ax3l Has a number of valid points.

However, I still believe that this PR is incremental progress towards the GitHub issue that I submitted back in Sept. #9384

We can always go back an improve support of virtual package support for variant + versioned APIs.

@tgamblin This is holding up progress of work that is related to our QMCPACK ECP. You are probably busy getting ready for SC18, would you please either approve the PR or let me know what changes are needed. I need to make two additional PR that depend on this one before the end of 2018Q4.

I am not an expert on QMCPACK, but can't you just depend(s)_on a specific FFTW implementation for now in it or expose switching the FFTW backend via a variant?

Quickly approving and changing everyone elses FFTW dependency is not the road I recommend on a Friday before SC. Ah wait, you go for a new package name...

@naromero77: is the fftw-api vdep sufficient for you? If so then we could move forward with it since you’re currently the Jong consumer, and I think we could add more later as you suggest. (And fix variant forwarding through vdeps soonish)

@tgamblin Yes, that is sufficient for QMCPACK, QE, etc.

If you would like me to, I can also open up a separate GitHub issue -- call it "Support for Variants in Virtual Packages." (Though I was hesitant to add to the large pile of issues.)