Skip to content

Bootstrap clingo from binaries#22720

Merged
becker33 merged 32 commits intospack:developfrom
alalazo:features/bootstrap_from_binaries
Aug 18, 2021
Merged

Bootstrap clingo from binaries#22720
becker33 merged 32 commits intospack:developfrom
alalazo:features/bootstrap_from_binaries

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Apr 1, 2021 •
edited
Loading

fixes #22606
fixes #23946
fixes #24417
fixes #24450
fixes #25164
closes #23898

Modifications:

  • Extend bootstrapping capabilities to use buildcaches
  • Allow to configure the order of methods tried for bootstrapping from configuration files
  • Allow to trust or untrust bootstrapping methods from cli
  • Add a command to list all known bootstrapping methods and their trust levels
  • Bootstrap from binaries in GA linux tests
  • Extend bootstrapping tests to buildcaches
  • Add documentation on the new spack bootstrap subcommands

To try this PR on x86-64, aarch64 or ppc64le machines:

$ cd <spack-repo>
$ git fetch upstream pull/22720/head:PR/22720
$ git checkout PR/22720

At that point you should be able to do something like:

$ spack find -b
==> Showing internal bootstrap store at "/home/culpo/.spack/bootstrap/store"
==> 0 installed packages

$ spack bootstrap list
[ shows list of known bootstrapping methods,  "github-actions" is TRUSTED ]

$ spack solve zlib
==> Best of 0 answers.
==> Optimization: [0, 0, 0, 0, 0, 1, 0, 0, 0]
[email protected]%[email protected]+optimize+pic+shared arch=linux-ubuntu18.04-broadwell

$ spack find -b
==> Showing internal bootstrap store at "/home/culpo/.spack/bootstrap/store"
==> 2 installed packages
-- linux-rhel5-x86_64 / [email protected] -------------------------------
clingo-bootstrap@spack  [email protected]

@alalazo alalazo added bootstrap Anything that has to do with Spack building its own dependencies. concretization labels Apr 1, 2021
@tgamblin tgamblin requested review from becker33 and tgamblin April 2, 2021 07:10
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch 2 times, most recently from ebd3d34 to 837babf Compare April 6, 2021 13:58
This was referenced Apr 7, 2021
Closed
Closed
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch 5 times, most recently from e097658 to 3d381a2 Compare April 16, 2021 12:54
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch 2 times, most recently from 1b1708c to a0c3b0d Compare April 21, 2021 13:02
@alalazo alalazo marked this pull request as ready for review April 21, 2021 14:06
This was referenced Apr 21, 2021
Closed
Closed
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch from a0c3b0d to 562e2f8 Compare April 30, 2021 15:55
@alalazo alalazo mentioned this pull request May 2, 2021
Merged
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch from 562e2f8 to a77fee4 Compare May 11, 2021 15:17
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch from a77fee4 to 45aa79f Compare May 24, 2021 08:31
This was referenced May 25, 2021
Closed
Closed
Closed
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch from 45aa79f to a20efbf Compare May 28, 2021 16:15
This was referenced May 28, 2021
Merged
Merged
alalazo added 21 commits August 13, 2021 17:46
@alalazo
Raise a RuntimeError if sha256 doesn't match
56a9bc8 
Demote verification message to debug level
@alalazo
Trust/untrust reports an error if more than one method matches
3c3f974 
Added unit tests to verify the behavior
@alalazo
Add a schema for bootstrap:trusted
9e169eb
@alalazo
Add more comment to defaults/bootstrap.yaml
bbfb109
@alalazo
Use target family to reconstruct compiler entries
10bc6f8
@alalazo
Use try/except block to be more defensive in case of unexpected errors
a268556
@alalazo
Require the old concretizer only for clingo
34847ae
@alalazo
Refactor bootstrapper classes to not call each other
61c32db
@alalazo
Be more defensive if bootstrapping methods fail because of bad config…
f8efbef 
…uration
@alalazo
Fix issue with scope=None and spack.config.set
aa39c51
@alalazo
Remove the adapter class for checking Spack's store
d89805c
@alalazo
Remove vestigial code related to the _StoreBootstrapper
6e7818e
@alalazo
Rename an argument for consistency with other functions
ef61ce8
@alalazo
Raise a more specific exception if a binary doesn't pass the checksum
c0a0632
@alalazo
Add a public url to the buildcache mirror, trust it by default
b4bd858 
Use buildcache in unit tests too
@alalazo
Update clingo.json, fix bootstrap workflow
7fc3fa1
@alalazo
Add binary tests for MacOS 🤞
6a9553b
@alalazo
Add binary tests for Ubuntu
975dfe6
@alalazo
Add binary tests for Ubuntu
2280e57
@alalazo
Add documentation
5d4b867
@alalazo
Add a note on bash
a355a53
@alalazo alalazo force-pushed the features/bootstrap_from_binaries branch from 861e474 to a355a53 Compare August 13, 2021 15:48
@alalazo alalazo mentioned this pull request Aug 18, 2021
Closed
3 tasks
@becker33 becker33 merged commit 4318ceb into spack:develop Aug 18, 2021
@alalazo alalazo deleted the features/bootstrap_from_binaries branch August 18, 2021 18:38
@alalazo
Copy link
Copy Markdown
Member Author

alalazo commented Aug 18, 2021

Thanks @becker33 !

@haampie
Copy link
Copy Markdown
Member

haampie commented Aug 18, 2021

nice!

@alalazo alalazo mentioned this pull request Aug 23, 2021
Closed
3 tasks
@tgamblin tgamblin mentioned this pull request Oct 25, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@becker33 becker33 becker33 approved these changes

@tgamblin tgamblin Awaiting requested review from tgamblin

Assignees

@tgamblin tgamblin

@becker33 becker33

Labels

binary-packages bootstrap Anything that has to do with Spack building its own dependencies. commands concretization defaults documentation Improvements or additions to documentation python sbang tests General test capability(ies) update-package workflow

Projects

None yet

Milestone

No milestone

6 participants

@alalazo @bvanessen @becker33 @christoph-conrads @haampie @tgamblin