Skip to content

concretizer:reuse:true w/ binary cache: concretizer fails if even one package in cache is not known to local spack? #36847

@eugeneswalker

Description

@eugeneswalker

Steps to reproduce

Trying to concretize a simple spec fails with E4S 23.02 build cache and concretizer:reuse:true set.

Here reproduced using:

  • Spack v0.19.1
  • ecpe4s/ubuntu20.04-runner-x86_64:2023-03-01 Docker container image
  • build cache: https://cache.e4s.io/23.02

All binaries @ https://cache.e4s.io/23.02 were made using Spack branch e4s-23.02, which is essentially spack@develop @ 50691cc (Thu Feb 16 11:57:26 2023 +0100).

$> git clone https://github.com/spack/spack
$> (cd spack && git checkout v0.19.1)
$> . spack/share/spack/setup-env.sh

$> spack mirror add E4S https://cache.e4s.io/23.02
$> spack buildcache keys -it
==> Fetching https://cache.e4s.io/23.02/build_cache/_pgp/25645FA2B218FE55B4EF649E4345F04B40005581.pub
gpg: key 4345F04B40005581: public key "University of Oregon - E4S" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: inserting ownertrust of 6

$> spack config add concretizer:reuse:true

$> spack spec zlib
==> Error: Package 'py-hatch-nodejs-version' not found.
You may need to run 'spack clean -m'.

With trace:

$> spack -d spec zlib
...
==> [2023-04-13-15:38:18.863141] UnknownPackageError: Package 'py-hatch-nodejs-version' not found.
    You may need to run 'spack clean -m'.
==> [2023-04-13-15:38:18.863199] Error: Package 'py-hatch-nodejs-version' not found.
You may need to run 'spack clean -m'.
Traceback (most recent call last):
  File "/spack/lib/spack/spack/repo.py", line 1357, in get_pkg_class
    module = importlib.import_module(fullname)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'spack.pkg.builtin.py-hatch-nodejs-version'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/spack/lib/spack/spack/main.py", line 1007, in main
    return _main(argv)
  File "/spack/lib/spack/spack/main.py", line 962, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "/spack/lib/spack/spack/main.py", line 990, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "/spack/lib/spack/spack/main.py", line 639, in _invoke_command
    return_val = command(parser, args)
  File "/spack/lib/spack/spack/cmd/spec.py", line 103, in spec
    concretized_specs = spack.cmd.parse_specs(args.specs, concretize=True)
  File "/spack/lib/spack/spack/cmd/__init__.py", line 228, in parse_specs
    spec.concretize(tests=tests)  # implies normalize
  File "/spack/lib/spack/spack/spec.py", line 2941, in concretize
    self._new_concretize(tests)
  File "/spack/lib/spack/spack/spec.py", line 2916, in _new_concretize
    result = solver.solve([self], tests=tests)
  File "/spack/lib/spack/spack/solver/asp.py", line 2438, in solve
    result, _, _ = self.driver.solve(setup, specs, reuse=reusable_specs, output=output)
  File "/spack/lib/spack/spack/solver/asp.py", line 708, in solve
    answers = builder.build_specs(tuples)
  File "/spack/lib/spack/spack/solver/asp.py", line 2304, in build_specs
    spack.spec.Spec.inject_patches_variant(root)
  File "/spack/lib/spack/spack/spec.py", line 2833, in inject_patches_variant
    pkg_deps = dspec.parent.package_class.dependencies
  File "/spack/lib/spack/spack/spec.py", line 1647, in package_class
    return spack.repo.path.get_pkg_class(self.fullname)
  File "/spack/lib/spack/spack/repo.py", line 981, in get_pkg_class
    return self.repo_for_pkg(pkg_name).get_pkg_class(pkg_name)
  File "/spack/lib/spack/spack/repo.py", line 1359, in get_pkg_class
    raise UnknownPackageError(pkg_name)
spack.repo.UnknownPackageError: Package 'py-hatch-nodejs-version' not found.
    You may need to run 'spack clean -m'.

Error message

No response

Information on your system

  • Spack: 0.19.1 (5e0d210)
  • Python: 3.8.10
  • Platform: linux-ubuntu20.04-cascadelake
  • Concretizer: clingo

CC @wspear @vicentebolea @alalazo

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions