Skip to content

Make internal concretization API more stable#47975

Open
alalazo wants to merge 4 commits intospack:developfrom
alalazo:hotfix/hash-computation
Open

Make internal concretization API more stable#47975
alalazo wants to merge 4 commits intospack:developfrom
alalazo:hotfix/hash-computation

Conversation

@alalazo
Copy link
Copy Markdown
Member

@alalazo alalazo commented Dec 7, 2024

fixes #47973

This fixes the inconsistency in hash computation observed on subsequent deconcretizations / reconcretizations of specs using the internal API.

Modifications:

  • Never cache dag_hash and process_hash on abstract specs
  • Simplify _mark_concrete logic, by removing a check

@spackbot-app spackbot-app bot added core PR affects Spack core functionality tests General test capability(ies) labels Dec 7, 2024
Comment on lines -2875 to -2964
if (not value) and self.concrete and self.installed:
return
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not sure why this was added. We need to double check if it was a performance fix, and if it is still needed. Given the type of checks, it's possibly related to the DB.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also have an issue where we don't enforce having the package hash in the node dict correctly, not really sure what to do about this:

spack/lib/spack/spack/spec.py

Lines 2235 to 2240 in 422f829

if (
self._concrete
and hash.package_hash
and hasattr(self, "_package_hash")
and self._package_hash
):

Basically, if a hash needs the package hash, but we don't have the attribute, we'll just skip it. This seems incorrect in general.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haampie What do you think about #47975 (comment) ? In my opinion the right thing to do would be to store the provenance of the package hash, which could be:

  1. Absent ("old" specs)
  2. Read from specfile, etc. (computed once upon a time, should be considered a constant that is difficult to be re-computed again)
  3. "Just" computed

The idea is that:

  • Only case 1 should be allowed not to have a package hash.
  • We probably never need to compute a hash for case 1 and 2, since we'll be reading the hash values from specfile
  • Only case 3 should drop the cached hash values on a call to Spec.clear_caches, since for the other cases the values are just constant that we might not be able to re-compute with the current Spack

I wonder if it is worth to refactor in this sense, instead of doing what we do now (if an attribute is not there, assume we are dealing with an old spec format).

@tgamblin tgamblin self-requested a review December 10, 2024 17:26
@alalazo alalazo force-pushed the hotfix/hash-computation branch from 7cce739 to 6f59ba2 Compare January 16, 2025 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistency in internal hash-related API

2 participants