Skip to content

ASP solver: two unit tests failing on MacOS #21215

@alalazo

Description

@alalazo

Running the unit test suite reports two failures, both related to pickling objects, when testing with the new concretizer. The errors do not occur with the original concretizer.

Steps to reproduce the issue

Setup clingo to be able to test the new concretizer. Then:

$ SPACK_TEST_SOLVER=clingo spack unit-test lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install lib/spack/spack/test/install.py::test_installed_upstream

Error Message

The error message is the following:

SPACK_TEST_SOLVER=clingo spack unit-test ...
% SPACK_TEST_SOLVER=clingo spack unit-test lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install lib/spack/spack/test/install.py::test_installed_upstream
=========================================================================================== test session starts ============================================================================================
platform darwin -- Python 3.8.5, pytest-3.2.5, py-1.4.34, pluggy-0.4.0
rootdir: /Users/culpo/PycharmProjects/spack, inifile: pytest.ini
collected 2 items                                                                                                                                                                                           

lib/spack/spack/test/cmd/deprecate.py F
lib/spack/spack/test/install.py F
========================================================================================= short test summary info ==========================================================================================
FAIL lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install[mock_archive0]
FAIL lib/spack/spack/test/install.py::test_installed_upstream[mock_archive0]

======================================================================================== slowest 20 test durations =========================================================================================
1.78s call     lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install[mock_archive0]
0.34s call     lib/spack/spack/test/install.py::test_installed_upstream[mock_archive0]
0.31s setup    lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install[mock_archive0]
0.01s setup    lib/spack/spack/test/install.py::test_installed_upstream[mock_archive0]
0.01s teardown lib/spack/spack/test/cmd/deprecate.py::test_deprecate_install[mock_archive0]
0.00s teardown lib/spack/spack/test/install.py::test_installed_upstream[mock_archive0]
================================================================================================= FAILURES =================================================================================================
__________________________________________________________________________________ test_deprecate_install[mock_archive0] ___________________________________________________________________________________

mock_packages = <spack.repo.RepoPath object at 0x102249220>
mock_archive = Archive(url='file:///private/var/folders/gy/g0z3m6dn0nz53nsttz2q0flm0000gn/T/pytest-of-culpo/pytest-84/mock-archive-di...sttz2q0flm0000gn/T/pytest-of-culpo/pytest-84/mock-archive-dir0/spack-src.tar.gz', expanded_archive_basedir='spack-src')
mock_fetch = None, install_mockery = None

    def test_deprecate_install(mock_packages, mock_archive, mock_fetch,
                               install_mockery):
        """Tests that the ```-i`` option allows us to deprecate in favor of a spec
        that is not yet installed."""
        install('[email protected]')
    
        to_deprecate = spack.store.db.query()
        assert len(to_deprecate) == 1
    
>       deprecate('-y', '-i', '[email protected]', '[email protected]')

lib/spack/spack/test/cmd/deprecate.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lib/spack/spack/main.py:545: in __call__
    self.returncode = _invoke_command(
lib/spack/spack/main.py:490: in _invoke_command
    return_val = command(parser, args)
lib/spack/spack/cmd/deprecate.py:129: in deprecate
    dcate.package.do_deprecate(dcator, link_fn)
lib/spack/spack/package.py:2229: in do_deprecate
    deprecator.package.do_install()
lib/spack/spack/package.py:1697: in do_install
    builder.install()
lib/spack/spack/installer.py:1534: in install
    self._install_task(task)
lib/spack/spack/installer.py:1122: in _install_task
    spack.build_environment.start_build_process(
lib/spack/spack/build_environment.py:945: in start_build_process
    serialized_pkg = spack.subprocess_context.PackageInstallContext(pkg)
lib/spack/spack/subprocess_context.py:69: in __init__
    self.serialized_pkg = serialize(pkg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <spack.pkg.builtin.mock.libelf.Libelf object at 0x102c253d0>

    def serialize(obj):
        serialized_obj = io.BytesIO()
>       pickle.dump(obj, serialized_obj)
E       _pickle.PicklingError: Can't pickle <class 'llnl.util.lang.Spec'>: attribute lookup Spec on llnl.util.lang failed

lib/spack/spack/subprocess_context.py:43: PicklingError
------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------
==> Fetching file:///private/var/folders/gy/g0z3m6dn0nz53nsttz2q0flm0000gn/T/pytest-of-culpo/pytest-84/mock-archive-dir0/spack-src.tar.gz
==> libelf: Executing phase: 'install'
==> libelf: Successfully installed libelf-0.8.10-gstytvv2sv35kdf6yw2uhg4wcankkprh
  Fetch: 0.02s.  Build: 0.47s.  Total: 0.49s.
[+] /private/var/folders/gy/g0z3m6dn0nz53nsttz2q0flm0000gn/T/pytest-of-culpo/pytest-84/test_deprecate_install_mock_ar0/opt/test-debian6-core2/gcc-4.5.0/libelf-0.8.10-gstytvv2sv35kdf6yw2uhg4wcankkprh
------------------------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------------------------
/Users/culpo/PycharmProjects/spack/lib/spack/spack/solver/concretize.lp:153:42-77: info: atom does not occur in any rule head:
  possible_provider(Package,Virtual)

/Users/culpo/PycharmProjects/spack/lib/spack/spack/solver/concretize.lp:153:42-77: info: atom does not occur in any rule head:
  possible_provider(Package,Virtual)

__________________________________________________________________________________ test_installed_upstream[mock_archive0] __________________________________________________________________________________

tmpdir_factory = <_pytest.tmpdir.TempdirFactory object at 0x101fefb20>, install_mockery = None, mock_fetch = None, gen_mock_layout = <function gen_mock_layout.<locals>.create_layout at 0x1021b73a0>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x102970730>

    def test_installed_upstream(tmpdir_factory, install_mockery, mock_fetch,
                                gen_mock_layout, monkeypatch):
        """Check that when a dependency package is recorded as installed in
           an upstream database that it is not reinstalled.
        """
        mock_db_root = str(tmpdir_factory.mktemp('mock_db_root'))
        prepared_db = spack.database.Database(mock_db_root)
    
        upstream_layout = gen_mock_layout('/a/')
    
        dependency = spack.spec.Spec('dependency-install')
        dependency.concretize()
        prepared_db.add(dependency, upstream_layout)
    
        downstream_db_root = str(
            tmpdir_factory.mktemp('mock_downstream_db_root'))
        db_for_test = spack.database.Database(
            downstream_db_root, upstream_dbs=[prepared_db])
        monkeypatch.setattr(spack.store, 'db', db_for_test)
        dependent = spack.spec.Spec('dependent-install')
        dependent.concretize()
    
        new_dependency = dependent['dependency-install']
        assert new_dependency.package.installed_upstream
        assert (new_dependency.prefix ==
                upstream_layout.path_for_spec(dependency))
    
>       dependent.package.do_install()

lib/spack/spack/test/install.py:246: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lib/spack/spack/package.py:1697: in do_install
    builder.install()
lib/spack/spack/installer.py:1534: in install
    self._install_task(task)
lib/spack/spack/installer.py:1122: in _install_task
    spack.build_environment.start_build_process(
lib/spack/spack/build_environment.py:945: in start_build_process
    serialized_pkg = spack.subprocess_context.PackageInstallContext(pkg)
lib/spack/spack/subprocess_context.py:69: in __init__
    self.serialized_pkg = serialize(pkg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <spack.pkg.builtin.mock.dependent-install.DependentInstall object at 0x1020ce1f0>

    def serialize(obj):
        serialized_obj = io.BytesIO()
>       pickle.dump(obj, serialized_obj)
E       _pickle.PicklingError: Can't pickle <class 'llnl.util.lang.Spec'>: attribute lookup Spec on llnl.util.lang failed

lib/spack/spack/subprocess_context.py:43: PicklingError
------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------
[+] /private/var/folders/gy/g0z3m6dn0nz53nsttz2q0flm0000gn/T/pytest-of-culpo/pytest-84/test_installed_upstream_mock_a0/a/dependency-install
==> Installing dependent-install-2.0-erjlafcwgpyioat5tfiedpy6nu3ohp5h
==> No binary for dependent-install-2.0-erjlafcwgpyioat5tfiedpy6nu3ohp5h found: installing from source
------------------------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------------------------
/Users/culpo/PycharmProjects/spack/lib/spack/spack/solver/concretize.lp:153:42-77: info: atom does not occur in any rule head:
  possible_provider(Package,Virtual)

/Users/culpo/PycharmProjects/spack/lib/spack/spack/solver/concretize.lp:153:42-77: info: atom does not occur in any rule head:
  possible_provider(Package,Virtual)

==> Error: Failed to install dependent-install due to PicklingError: Can't pickle <class 'llnl.util.lang.Spec'>: attribute lookup Spec on llnl.util.lang failed
========================================================================================= 2 failed in 2.84 seconds =========================================================================================

Information on your system

  • Spack: 0.16.0-918-ec7caefc69
  • Python: 3.8.5
  • Platform: darwin-bigsur-cannonlake
  • Concretizer: clingo

The exact version of the OS is 11.1 (20C69)

Additional information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions