Skip to content

Incomplete computation of installed dependents #11983

Closed
#28673
Closed
Incomplete computation of installed dependents#11983
#28673
Assignees
alalazo
Labels
bugSomething isn't workingdependenciesimpact-mediumspecs
@alalazo

Description

@alalazo
alalazo
opened on Jul 11, 2019

Follow-up of #3690 (comment). Basically everything that relies on computing dependents is broken when a spec has more than one dependent with the same name.

Steps to reproduce the issue

The issue can be reproduced with e.g. Spack at d3be42f. Let's start by installing 2 different versions of hdf5 depending on the same zlib:

$ spack install --fake hdf5~mpi+fortran hdf5~mpi~fortran

Let's then check that they actually both depend on the same zlib and they have different hashes:

$ spack find -ld hdf5
==> 2 installed packages
-- linux-ubuntu18.04-x86_64 / [email protected] -------------------------
xyhss36    [email protected]
tn4qvs7        ^[email protected]

4f3j7km    [email protected]
tn4qvs7        ^[email protected]

Asking which are the installed dependents of zlib will give the wrong answer an skip one of the two specs:

$ spack dependents -i zlib
==> Dependents of [email protected]%[email protected]/tn4qvs7
-- linux-ubuntu18.04-x86_64 / [email protected] -------------------------
xyhss36 [email protected]

This is ultimately due to how dependents are computed in spack.spec.Spec:

spack/lib/spack/spack/spec.py

Lines 1063 to 1072 in c2de255

def _add_dependency(self, spec, deptypes):
"""Called by the parser to add another spec as a dependency."""
if spec.name in self._dependencies:
raise DuplicateDependencyError(
"Cannot depend on '%s' twice" % spec)
# create an edge and add to parent and child
dspec = DependencySpec(self, spec, deptypes)
self._dependencies[spec.name] = dspec
spec._dependents[self.name] = dspec

by constructing a map from the package name to the branch connecting it to the current spec. When there is more than one dependent with the same name only the last dependent added will be recorded.

This might be fine if we are working on a single tree DAG (where we are ensured there's at most one spec with the same name), but it is not if we are analyzing the entire DAG of all the installed applications. For instance the method:

spack/lib/spack/spack/database.py

Lines 878 to 911 in d3be42f

def installed_relatives(self, spec, direction='children', transitive=True,
deptype='all'):
"""Return installed specs related to this one."""
if direction not in ('parents', 'children'):
raise ValueError("Invalid direction: %s" % direction)
relatives = set()
for spec in self.query(spec):
if transitive:
to_add = spec.traverse(
direction=direction, root=False, deptype=deptype)
elif direction == 'parents':
to_add = spec.dependents(deptype=deptype)
else: # direction == 'children'
to_add = spec.dependencies(deptype=deptype)
for relative in to_add:
hash_key = relative.dag_hash()
upstream, record = self.query_by_spec_hash(hash_key)
if not record:
reltype = ('Dependent' if direction == 'parents'
else 'Dependency')
msg = ("Inconsistent state! %s %s of %s not in DB"
% (reltype, hash_key, spec.dag_hash()))
if self._fail_when_missing_deps:
raise MissingDependenciesError(msg)
tty.warn(msg)
continue
if not record.installed:
continue
relatives.add(relative)
return relatives

will return wrong results when we ask to analyze in the parents direction.

Error Message

There's no error message, just the information on dependents of an installed spec is computed wrongly. This lead for instance to issues like #3690 or #5637.

Information on your system

The platform is not relevant for this bug.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdependenciesimpact-mediumspecs

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions