Summary from DMs:

• Package API version needs to be bumped
• The following makes little sense: x %%y implies x %(y %y).
  • If %% applies only to the link(/run) sub-dag, this is not a problem in practice. The %%y is by convention a build dependency, so foo %%gcc@15 would not imply foo %(gcc@15 %gcc@15).
• Users still need to write %%c,cxx,fortran=compiler. Don't heuristically expand %%compiler to %[when=%c]c=compiler ... for all languages, cause languages are (or should be) defined by the repository and may evolve.
• Without virtuals, %%cmake@3 could mean %[when=%cmake]cmake@3, cause you already branch on whether virtuals are set.

Other thoughts:

An issue remains when clang is used as a compiler but also libLLVM is used as a link dep. With the above, %%c,cxx=clang would still imply llvm %c,cxx=clang, which is undesirable. Same with libgccjit.

• Maybe a pragmatic workaround is to not enforce %%c,cxx=clang on dependencies that are providers of c or cxx, even if they are link/run deps.
• Or, probably too involved, you could follow edges to non-c/cxx providers (instead of following link/run edges).

Certain things related to compiler dependencies may work now as a happy coincidence: compilers are either externals or reused. That might not always be the case.

Still missing docs, but should be ready for review

Documentation here is minimal, and a small refactor has been extracted in #51474. If people agree I would keep it like it is in this PR, and have another dedicated PR to add more examples in the environment section to explain how the feature can be used in common scenarios.

The use cases would be those in the tests added to cmd/env.py.

Testing this locally. I'm hitting a strange error:

$ spack --backtrace add trilinos%%mixed_llvm
Traceback (most recent call last):
  File "/scratch/psakiev/spack-dev/environments/spack/bin/spack", line 49, in <module>
    sys.exit(main())
             ^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/main.py", line 1095, in main
    return _main(argv)
           ^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/main.py", line 1047, in _main
    return finish_parse_and_run(parser, cmd_name, args, env_format_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/main.py", line 1078, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/main.py", line 591, in _invoke_command
    return_val = command(parser, args)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/cmd/add.py", line 32, in add
    if not env.add(spec, args.list_name):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/environment/environment.py", line 1320, in add
    existing = str(spec) in list_to_change.yaml_list
               ^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/spec.py", line 4420, in __str__
    return self._str(color=False)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/spec.py", line 4416, in _str
    return self._long_spec(color=color)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/spec.py", line 4359, in _long_spec
    return f"{self.format(color=color)} {self._format_dependencies(color=color)}".strip()
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/psakiev/spack-dev/environments/spack/lib/spack/spack/spec.py", line 4315, in _format_dependencies
    for edge in sorted(direct, key=lambda x: x.spec.name):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'str' and 'NoneType'

I'm working on trying to debug, but figured I'd post early incase it triggers some thoughts on a root cause before I find it.

Update 1:

I wonder if this is due to how spack add is functioning for a toolchain:

$ spack add trilinos%mixed_llvm
==> Adding trilinos %[when='%fortran'] fortran=gcc %[when='%c'] c=clang %[when='%cxx'] cxx=clang to environment /scratch/psakiev/spack-dev/environments/demo-envs/percent-percent

The expansion is probably not parsable since we are communicating config, not specs.

Update 2:

Even when I manually put the trilinos%%mixed_llvm in as the root spec I'm still seeing this error.

@psakievich I couldn't reproduce. I just copied the example I have in the description, and tried to concretize:

@alalazo I am now able to reproduce, and good news, use with all of sierra. Concretization is notably slower:

• %gcc: 0min 35sec (used a single root inside sierra bundle)
• %%gcc: 1min 28sec (used a single root inside sierra bundle)
• %llvm_fortran: 2min 43sec
• %%llvm_fortran: 3min 42sec

I'll test again with the newer commits I see.

Also noted that I can't apply compiler constraints or propagations to bundle packages:

==> SOLVING SPEC:
  sierra %gcc
==> Error: 'sierra %gcc' cannot depend on gcc`

==> SOLVING SPEC:
  sierra %%gcc
==> Error: 'sierra %%gcc' cannot depend on gcc

It's nice that toolchains seem to work, but they are quite a bit slower.

Update

Timings for %% about the same with f1d36afec8efae4c60207ab0b315d3f5b9a7a037

Using propagation is slow, due to the closures we have to compute to only propagate on the link/run subdag. I guess the good news is that when we don't use propagation, times are the same as before 😆 .

Also noted that I can't apply compiler constraints or propagations to bundle packages:

Well, that's how things work and wasn't changed in this PR. To propagate compiler constraints you have to use toolchains, or conditional dependencies directly.

Do you have a reproducer for the error in #51383 (comment) ?

Using propagation is slow, due to the closures we have to compute to only propagate on the link/run subdag. I guess the good news is that when we don't use propagation, times are the same as before 😆 .

Also noted that I can't apply compiler constraints or propagations to bundle packages:

Well, that's how things work and wasn't changed in this PR. To propagate compiler constraints you have to use toolchains, or conditional dependencies directly.

I figured as much, but it took me a couple of try's before I realized it.

Do you have a reproducer for the error in #51383 (comment) ?

I never did figure this one out. I recloned spack and started with your reproducer. I suspect I inadvertently switched to the wrong branch or something else user error related.

First the good news. This doesn't affect the concretization time, on our usual benchmark, when the option is not used. Tested against:

• Spack: 1.1.0.dev0 (b1ed39a)
• Builtin repo: spack/spack-packages@65fb65e
• Python: 3.13.2
• Platform: linux-ubuntu20.04-icelake

radiuss.pr.csv
radiuss.develop.csv

Now the bad news. I started seeing fluctuations with very long solve times on some scrambled output of py-cinemasci. This happens both on develop and on this PR (and that's why I didn't include the spec above). I'll investigate that, but I bet it's some package change that went in in the last couple of weeks 🫠