Skip to content

Inconsistency in internal hash-related API #47973

@alalazo

Description

@alalazo

Steps to reproduce

Run the following script:

from spack.spec import Spec

s = Spec("gmake~guile").concretized()
for _ in range(3):
    print(s.tree(hashes=True))
    print()
    s._mark_concrete(False)
    s._finalize_concretization()

The expected result would be to see 3 times the same hashes, since we use internal API to deconcretize a spec. The outcome is different though:

Screenshot from 2024-12-07 11-22-22

Error message

The issue is caused by a mixture of possibly wrong logic and caches. The first time we compute the hash we get the result we expect. In the following cases, when calling:

spack/lib/spack/spack/spec.py

Lines 2900 to 2913 in 05acd29

def _mark_concrete(self, value=True):
"""Mark this spec and its dependencies as concrete.
Only for internal use -- client code should use "concretize"
unless there is a need to force a spec to be concrete.
"""
# if set to false, clear out all hashes (set to None or remove attr)
# may need to change references to respect None
for s in self.traverse():
if (not value) and s.concrete and s.installed:
continue
elif not value:
s.clear_caches()
s._mark_root_concrete(value)

with value=False, we first clear the caches, then execute:

spack/lib/spack/spack/spec.py

Lines 2873 to 2879 in 05acd29

def _mark_root_concrete(self, value=True):
"""Mark just this spec (not dependencies) concrete."""
if (not value) and self.concrete and self.installed:
return
self._normal = value
self._concrete = value
self._validate_version()

The first part of the issue is that this call recreates the caches we just deleted, via the Spec.installed property. It is not clear to me why we have:

spack/lib/spack/spack/spec.py

Lines 2875 to 2876 in 05acd29

if (not value) and self.concrete and self.installed:
return

or what this check even means.

When we recreate the hashes, we execute:

node_dict = self.to_node_dict(hash=hash)

and that function does not enforce having a package_hash for hashes that need it:

spack/lib/spack/spack/spec.py

Lines 2235 to 2240 in 05acd29

if (
self._concrete
and hash.package_hash
and hasattr(self, "_package_hash")
and self._package_hash
):

Above we see that if an hash requires the package hash, but we don't have it, we simply skip adding it - which results in the wrong dict being used.

Information on your system

  • Spack: 0.24.0.dev0 (f181ac1)
  • Python: 3.13.0
  • Platform: linux-ubuntu20.04-icelake

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

Metadata

Metadata

Assignees

Labels

Type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions