Skip to content

concretizer: update reuse: default to True#41302

Merged
alalazo merged 7 commits intodevelopfrom
settings/default-reuse-not-fresh-roots
Apr 23, 2024
Merged

concretizer: update reuse: default to True#41302
alalazo merged 7 commits intodevelopfrom
settings/default-reuse-not-fresh-roots

Conversation

@becker33
Copy link
Copy Markdown
Member

We've seen a rash of issues with the default reuse parameters for the concretizer.

Specifically, many users have the following workflow:

  1. Install spec foo+bar
  2. Push foo to buildcache
  3. Spack install foo

In that workflow, foo should install as foo+bar from binary. That will not necessarily happen with the current defaults.

This gets even worse in the case of a shared binary cache (like the spack public caches) that may have newer versions of the dependencies. In that case, the install may not work from binary even without specifying a variant or without any non-default aspects for the package in the buildcache.

@tgamblin and I have come to the belief that the current default is less intuitive than pure reuse. The reuse: dependencies config options and the --reuse-deps flags are still available for users who want the fresh concretization of roots.

@spackbot-app spackbot-app bot added core PR affects Spack core functionality defaults labels Nov 28, 2023
@haampie
Copy link
Copy Markdown
Member

haampie commented Nov 28, 2023

I think it's pretty ambiguous which default is better.

Iterating with spack edit x spack install x is more convenient with current defaults.

So is spack checksum --add-to-package x spack install x.

tgamblin
tgamblin previously approved these changes Dec 11, 2023
@tgamblin
Copy link
Copy Markdown
Member

please add to #30634

@tgamblin tgamblin changed the title concretizer:reuse update default to True concretizer: update reuse: default to True Dec 21, 2023
@spackbot-app spackbot-app bot added the tests General test capability(ies) label Dec 22, 2023
@michaelkuhn
Copy link
Copy Markdown
Member

I agree with @haampie, the usual development workflows are pretty annoying with reuse: True.

Moreover, this breaks the upgrade workflow:

$ spack install xyz
# Update Spack
$ spack mark -ai
$ spack install xyz
$ spack gc

With reuse: True, the second install will usually simply reuse the old version.

@tgamblin
Copy link
Copy Markdown
Member

Ok given the lack of consensus, I'm proposing we move this to a more detailed discussion. I started one here:

The gist of it is that I think both consumer workflows and dev/upgrade/iterative workflows are important, so we should have command that serve them both. So I'm proposing to split some of this out to spack upgrade.

Please throw your thoughts over there.

@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Jan 8, 2024

@spackbot fix style

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Jan 8, 2024

Let me see if I can fix that for you!

@spackbot-app
Copy link
Copy Markdown

spackbot-app bot commented Jan 8, 2024

I was able to run spack style --fix for you!

spack style --fix
==> Running style checks on spack
  selected: isort, black, flake8, mypy
==> Modified files
  lib/spack/spack/solver/asp.py
  lib/spack/spack/test/cmd/common/arguments.py
  lib/spack/spack/test/concretize.py
==> Running isort checks
  isort checks were clean
==> Running black checks
reformatted lib/spack/spack/test/concretize.py
All done! ✨ 🍰 ✨
1 file reformatted, 2 files left unchanged.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> Running mypy checks
Success: no issues found in 610 source files
  mypy checks were clean
==> spack style checks were clean
Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with style fixes.

@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Jan 9, 2024

@tgamblin finally tracked down all the tests that relied on the old behavior and fixed them.

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented Jan 9, 2024

Based on the conflicting view here, I think we should wait to merge this for an upgrade command, so please weigh in on the discussion. It seems like we're pretty close to a consensus over there -- there is a debate about how aggressive upgrade should be.

#41882

@haampie
Copy link
Copy Markdown
Member

haampie commented Apr 23, 2024

I changed my mind about this after #43190.

Build caches are much more useful, and the experience is more apt-get install ... like. But only if roots are reused.

I would say let's change this default for 0.22

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Apr 23, 2024

With libc merged in #43190, and other improvements that make it possible to install specs without having their compiler configured locally, it makes more sense to have --reuse as a default. This would allow, for instance:

$ spack install gcc

to work without a compiler currently configured, if a binary is in a registered buildcache.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Apr 23, 2024

Since the consensus changed, I'm going to merge this one. Checked with @michaelkuhn in DM

@alalazo alalazo merged commit 978c20f into develop Apr 23, 2024
@alalazo alalazo deleted the settings/default-reuse-not-fresh-roots branch April 23, 2024 15:42
tldahlgren pushed a commit to tldahlgren/spack that referenced this pull request Apr 23, 2024
stephenmsachs pushed a commit to stephenmsachs/spack that referenced this pull request Apr 29, 2024
teaguesterling pushed a commit to teaguesterling/spack that referenced this pull request Jun 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality defaults tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants