This was missed while backporting the new `spack info` command from #40326.

Variants should be sorted by name when invoking `spack info --variants-by-name`.

This was missed while backporting the new `spack info` command from #40326.

Variants should be sorted by name when invoking `spack info --variants-by-name`.

This was missed while backporting the new `spack info` command from #40326.

Variants should be sorted by name when invoking `spack info --variants-by-name`.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

There are a couple other changes here that might be of interest:

1. The `generator` variant in `CMakePackage` is now only defined when `build_system=cmake`.
2. `spack info` has been updated to support the new metadata layout.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

…41975)

Needed for spack#40326, which can changes the iteration order over package dependencies during concretization.

While clingo doesn't have this problem, the original concretizer (which we still use for bootstrapping) can be sensitive to iteration order when evaluating dependency constraints in `when` conditions. This can cause it to ignore conditional dependencies unless the dependencies in the condition are listed first in the package.

The issue was in the way the original concretizer would disconnect specs *every* time `normalize()` ran. When specs were disconnected, `^dependency` constraints wouldn't see the dependency in the dependency condition loop.

We now only only disconnect *all* dependencies at the start of `concretize()` and `normalize()`, and we disconnect any leftover dependents from replaced externals at the *end* of `normalize()`.  This trims stale connections while keeping the ones that are needed to trigger dependency conditions.

- [x] refactor `flat_dependencies()` to not disconnect the spec by default.
- [x] `flat_dependencies()` is never called with `copy=True` -- remove the `copy` kwarg.
- [x] disconnect only once at the beginning of `normalize()` or `concretize()`.
- [x] add a test that perturbs dependency iteration order to ensure this doesn't regress.
- [x] disconnect unused dependents at end of `normalize()`

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure of Variant metadata on
Packages from a single variant definition per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly) all modeled in the
   same way; and
2. fixes an issue where we would lose information about multiple variant
   definitions in a package (see #38302). We can now have, e.g., different defaults for
   the same variant in different versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously, the solver's job
   was to select value(s) for a single variant definition per name per package. Now, the
   solver needs to:

   a. Determine which variant definition should be used for a given node, which can depend
      on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected variant
      definition.

   The solver will always prefer the *last* matching variant definition based on declaration
   order in packages. This has the effect that variant definitions from derived classes are
   preferred over definitions from superclasses, and the last definition within the same
   class sticks. I think is intuitive b/c it matches python semantics. I hope this is fleshed
   out clearly in the solver logic.

2. The solver does a lot more validation on variant values at setup time now. In particular,
   it checks whether a variant value on a spec is valid given the other constraints on that
   spec.

3. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid given
   the conditions on variant definitions in packages. I haven't fixed those here but will
   open a separate PR to iterate on them.  I plan to make strict checking the defaults once
   all existing package issues are resolved. It's not clear to me that strict checking should
   be the default for the prevalidation done at solve time.

Signed-off-by: Todd Gamblin <[email protected]>

Continuing the work started in #40326, his changes the structure
of Variant metadata on Packages from a single variant definition
per name with a list of `when` specs:

```
name: (Variant, [when_spec, ...])
```

to a Variant definition per `when_spec` per name:

```
when_spec: { name: Variant }
```

With this change, everything on a package *except* versions is
 keyed by `when` spec. This:

1. makes things consistent, in that conditional things are (nearly)
   all modeled in the same way; and

2. fixes an issue where we would lose information about multiple
   variant definitions in a package (see #38302). We can now have,
   e.g., different defaults for the same variant in different
   versions of a package.

Some notes:

1. This required some pretty deep changes to the solver. Previously,
   the solver's job was to select value(s) for a single variant definition
   per name per package. Now, the solver needs to:

   a. Determine which variant definition should be used for a given node,
      which can depend on the node's version, compiler, target, other variants, etc.
   b. Select valid value(s) for variants for each node based on the selected
      variant definition.

   When multiple variant definitions are enabled via their `when=` clause, we will
   always prefer the *last* matching definition, by declaration order in packages. This
   is implemented by adding a `precedence` to each variant at definition time, and we
   ensure they are added to the solver in order of precedence.

   This has the effect that variant definitions from derived classes are preferred over
   definitions from superclasses, and the last definition within the same class sticks.
   This matches python semantics. Some examples:

    ```python
    class ROCmPackage(PackageBase):
        variant("amdgpu_target", ..., when="+rocm")

    class Hipblas(ROCmPackage):
        variant("amdgpu_target", ...)
    ```

   The global variant in `hipblas` will always supersede the `when="+rocm"` variant in
   `ROCmPackage`. If `hipblas`'s variant was also conditional on `+rocm` (as it probably
   should be), we would again filter out the definition from `ROCmPackage` because it
   could never be activated. If you instead have:

    ```python
    class ROCmPackage(PackageBase):
        variant("amdgpu_target", ..., when="+rocm")

    class Hipblas(ROCmPackage):
        variant("amdgpu_target", ..., when="+rocm+foo")
    ```

   The variant on `hipblas` will win for `+rocm+foo` but the one on `ROCmPackage` will
   win with `rocm~foo`.

   So, *if* we can statically determine if a variant is overridden, we filter it out.
   This isn't strictly necessary, as the solver can handle many definitions fine, but
   this reduces the complexity of the problem instance presented to `clingo`, and
   simplifies output in `spack info` for derived packages. e.g., `spack info hipblas`
   now shows only one definition of `amdgpu_target` where before it showed two, one of
   which would never be used.

2. Nearly all access to the `variants` dictionary on packages has been refactored to
   use the following class methods on `PackageBase`:
    * `variant_names(cls) -> List[str]`: get all variant names for a package
    * `has_variant(cls, name) -> bool`: whether a package has a variant with a given name
    * `variant_definitions(cls, name: str) -> List[Tuple[Spec, Variant]]`: all definitions
      of variant `name` that are possible, along with their `when` specs.
    * `variant_items() -> `: iterate over `pkg.variants.items()`, with impossible variants
      filtered out.

   Consolidating to these methods seems to simplify the code a lot.

3. The solver does a lot more validation on variant values at setup time now. In
   particular, it checks whether a variant value on a spec is valid given the other
   constraints on that spec. This allowed us to remove the crufty logic in
   `update_variant_validate`, which was needed because we previously didn't *know* after
   a solve which variant definition had been used. Now, variant values from solves are
   constructed strictly based on which variant definition was selected -- no more
   heuristics.

4. The same prevalidation can now be done in package audits, and you can run:

   ```
   spack audit packages --strict-variants
   ```

   This turns up around 18 different places where a variant specification isn't valid
   given the conditions on variant definitions in packages. I haven't fixed those here
   but will open a separate PR to iterate on them. I plan to make strict checking the
   defaults once all existing package issues are resolved. It's not clear to me that
   strict checking should be the default for the prevalidation done at solve time.

There are a few other changes here that might be of interest:

  1. The `generator` variant in `CMakePackage` is now only defined when `build_system=cmake`.
  2. `spack info` has been updated to support the new metadata layout.
  3.  split out variant propagation into its own `.lp` file in the `solver` code.
  4. Add better typing and clean up code for variant types in `variant.py`.
  5. Add tests for new variant behavior.

This was missed while backporting the new `spack info` command from spack#40326.

Variants should be sorted by name when invoking `spack info --variants-by-name`.

This was missed while backporting the new `spack info` command from spack#40326.

Variants should be sorted by name when invoking `spack info --variants-by-name`.