Fixes #38302.
Enables #43961.

Continuing the work started in #40326, his changes the structure of Variant metadata on Packages from a single variant definition per name with a list of when specs:

name: (Variant, [when_spec, ...])

to a Variant definition per when_spec per name:

when_spec: { name: Variant }

With this change, everything on a package except versions is keyed by when spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the same way; and
2. fixes an issue where we would lose information about multiple variant definitions in a package (see Variant when=: Change of Default for Single/Multi-Variant Transition #38302). We can now have, e.g., different defaults for the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job was to select value(s) for a single variant definition per name per package. Now, the solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant definition.

   When multiple variant definitions are enabled via their when= clause, we will always prefer the last matching definition, by declaration order in packages. This is implemented by adding a precedence to each variant at definition time, and we ensure they are added to the solver in order of precedence.

   This has the effect that variant definitions from derived classes are preferred over definitions from superclasses, and the last definition within the same class sticks. This matches python semantics. Some examples:

   class ROCmPackage(PackageBase):
       variant("amdgpu_target", ..., when="+rocm")
   
   class Hipblas(ROCmPackage):
       variant("amdgpu_target", ...)

   The global variant in hipblas will always supersede the when="+rocm" variant in ROCmPackage. If hipblas's variant was also conditional on +rocm (as it probably should be), we would again filter out the definition from ROCmPackage because it could never be activated. If you instead have:

   class ROCmPackage(PackageBase):
       variant("amdgpu_target", ..., when="+rocm")
   
   class Hipblas(ROCmPackage):
       variant("amdgpu_target", ..., when="+rocm+foo")

   The variant on hipblas will win for +rocm+foo but the one on ROCmPackage will win with rocm~foo.

   So, if we can statically determine if a variant is overridden, we filter it out. This isn't strictly necessary, as the solver can handle many definitions fine, but this reduces the complexity of the problem instance presented to clingo, and simplifies output in spack info for derived packages. e.g., spack info hipblas now shows only one definition of amdgpu_target where before it showed two, one of which would never be used.

2. Nearly all access to the variants dictionary on packages has been refactored to use the following class methods on PackageBase:

   • variant_names(cls) -> List[str]: get all variant names for a package
   • has_variant(cls, name) -> bool: whether a package has a variant with a given name
   • variant_definitions(cls, name: str) -> List[Tuple[Spec, Variant]]: all definitions of variant name that are possible, along with their when specs.
   • variant_items() -> : iterate over pkg.variants.items(), with impossible variants filtered out.

   Consolidating to these methods seems to simplify the code a lot.

3. The solver does a lot more validation on variant values at setup time now. In particular, it checks whether a variant value on a spec is valid given the other constraints on that spec. This allowed us to remove the crufty logic in update_variant_validate, which was needed because we previously didn't know after a solve which variant definition had been used. Now, variant values from solves are constructed strictly based on which variant definition was selected -- no more heuristics.

4. The same prevalidation can now be done in package audits, and you can run:

   spack audit packages --strict-variants
   

   This turns up around 18 different places where a variant specification isn't valid given the conditions on variant definitions in packages. I haven't fixed those here but will open a separate PR to iterate on them. I plan to make strict checking the defaults once all existing package issues are resolved. It's not clear to me that strict checking should be the default for the prevalidation done at solve time.

There are a few other changes here that might be of interest:

1. The generator variant in CMakePackage is now only defined when build_system=cmake.
2. spack info has been updated to support the new metadata layout.
3. split out variant propagation into its own .lp file in the solver code.
4. Add better typing and clean up code for variant types in variant.py.
5. Add tests for new variant behavior.