Skip to content

Fix bug getting specs from build caches#9600

Merged
gartung merged 1 commit intospack:developfrom
JavierCVilla:bugs/buildcache-spec
Feb 15, 2019
Merged

Fix bug getting specs from build caches#9600
gartung merged 1 commit intospack:developfrom
JavierCVilla:bugs/buildcache-spec

Conversation

@JavierCVilla
Copy link
Copy Markdown
Contributor

Use list rather than set when getting specs from YAML file (spec.yaml)

Context

Currently there seems to be cases where two specs, with different dag-hashes, share the same internal hash - returned by hash(spec). This produces some problems when installing from binaries and using the hash to specify the package to be installed, example:

spack  buildcache install /123456789

Let's say there are two YAML files with the same name, version and compiler but different hashes such as:

linux-centos7-x86_64-gcc-6.2.0-packageA-1.0-987654321-spec.yaml
linux-centos7-x86_64-gcc-6.2.0-packageA-1.0-123456789-spec.yaml

I don't fully understand how this is even possible, but there are cases where the specs produced out of these two YAML files can have the same internal hash.

As a consequence, using a set in the implementation of get_specs will consider only one of the two specs with the same internal hash but different dag-hash, thus leading to a problem when the user tries to install the excluded spec.

This could be also solved by implementing a __hash__ method in Spec (which would call to dag_hash() in order to use the same hash employed to concretize the spec), however this commit warns not to use such implementation due to inefficient side-effects.

I have tried to implement this __hash__ method and it certainly slows down the performance of the spack spec command, although not as much as before:

With __hash__ implemented on Spec

time spack spec dealii

real	0m9.371s
user	0m6.975s
sys	0m0.316s

Without:

real	0m7.244s
user	0m6.911s
sys	0m0.316s

The impact might be bigger on other packages so I would just use a list here rather than adding a hash method to the Spec class.

@tgamblin @gartung

@gartung
Copy link
Copy Markdown
Member

gartung commented Oct 22, 2018

@JavierCVilla #8451 might be of iterest.

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Oct 24, 2018

I don't fully understand how this is even possible

I think we should pin down the root cause of this, otherwise we would just be curing the symptom...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants