Steps to reproduce

TL;DR The approach we adopted since we added stacks to soft-enforce constraints in matrices result in the highly inconsistent behavior below. I think we should get rid of it to:

1. Have a more consistent behavior in environments
2. Avoid having lockfiles where user specs that don't intersect map to the same concrete spec

The good

If we try to concretize the following spack.yaml:

spack:
  specs:
  - zlib +shrd
  - zlib ~shrd
  unify: false
  view: false

we get the following error:

$ spack concretize
==> Starting concretization pool with 2 processes
==> Error: trying to set variant "shrd" in package "zlib", but the package has no such variant [happened during concretization of zlib+shrd]

which seems sensible: zlib has in fact no variant shrd.

The bad

If we apply this transformation, which should be equivalent:

spack:
  specs:
  - matrix:
    - [zlib]
    - [+shrd, ~shrd]
  unify: false
  view: false

we get instead:

$ spack concretize -f
==> Starting concretization pool with 2 processes
==> Environment concretized in 3.04 seconds
==> Concretized zlib+shrd
 -   ctm7p7o  [email protected]%[email protected]+optimize+pic+shared build_system=makefile arch=linux-ubuntu20.04-icelake
 -   3wudxnw      ^[email protected]%[email protected]~guile build_system=generic arch=linux-ubuntu20.04-icelake

==> Concretized zlib~shrd
 -   ctm7p7o  [email protected]%[email protected]+optimize+pic+shared build_system=makefile arch=linux-ubuntu20.04-icelake
 -   3wudxnw      ^[email protected]%[email protected]~guile build_system=generic arch=linux-ubuntu20.04-icelake

This is utterly confusing: the two specs result in the same concrete spec. The lockfile starts with:

"roots": [
    {
      "hash": "ctm7p7opscnjqbtptefifmthqm74x2gf",
      "spec": "zlib+shrd"
    },
    {
      "hash": "ctm7p7opscnjqbtptefifmthqm74x2gf",
      "spec": "zlib~shrd"
    }
  ],

so we effectively map two incompatible user specs to the same concrete spec that does not contain a required variant.

The ugly

Since the concrete spec is the same, one would expect the same behavior for:

spack:
  specs:
  - matrix:
    - [zlib]
    - [+shrd, ~shrd]
  unify: true
  view: false

but this one instead fails with:

$ spack concretize -f
==> Error: trying to set variant "shrd" in package "zlib", but the package has no such variant [happened during concretization of zlib~shrd]

Error message

See cases above. I think that:

1. All three cases should error consistently
2. We should enforce the requirement in a strict way, and avoid special casing unify:false + matrices

Information on your system

• Spack: 0.21.0.dev0 (3e51778)
• Python: 3.11.5
• Platform: linux-ubuntu20.04-icelake
• Concretizer: clingo

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output