Skip to content

spack develop clone with resources errors #25370

Open
Open
spack develop clone with resources errors#25370
Labels
bugSomething isn't workingdev-specsAnything related to dev-specs in environmentsimpact-lowsnl-core-teamIssue for SNL Spack developers
@psakievich

Description

@psakievich
psakievich
opened on Aug 11, 2021

Steps to reproduce

When trying to use spack develop with Trilinos an error occurs during the call to spack develop and the Trilinos spec is not added as a develop spec

spack env create -d . 
spacktivate .
spack add trilinos
spack develop trilinos@develop
==> Configuring spec trilinos@develop for development at path trilinos
==> Error: [Errno 17] File exists: '/projects/wind/spack-manager/environments/cee/test/trilinos/.git'

Further inspection of the spack.yaml shows no development spec has been added.
The Trilinos repo that is cloned looks fine and will work if you manually add the develop spec to the spack.yaml.

If you run spack develop trilinos@develop again it will complete without error and the develop information will be added to spack.yaml

The same behavior also occurs when using trilinos@master for the develop spec.

Error message

[psakiev@ascicgpu24 test2]$ spack -d develop trilinos@develop
==> [2021-08-11-14:49:43.245237] Imported develop from built-in commands
==> [2021-08-11-14:49:43.246412] Imported develop from built-in commands
==> [2021-08-11-14:49:43.254839] Configuring spec trilinos@develop for development at path trilinos
==> [2021-08-11-14:49:43.255758] Reading config file /projects/wind/spack-manager/spack/etc/spack/defaults/repos.yaml
==> [2021-08-11-14:49:43.259631] Reading config file /ascldap/users/psakiev/.spack/repos.yaml
==> [2021-08-11-14:49:43.261874] Reading config file /projects/wind/spack-manager/environments/cee/test/test2/spack.yaml
==> [2021-08-11-14:49:43.443007] Creating stage lock spack-stage-trilinos-develop-xjqqtbwx3wofw4lwwodf6vaehx5y5t7e
==> [2021-08-11-14:49:43.443580] Creating stage lock resource-dtk-xjqqtbwx3wofw4lwwodf6vaehx5y5t7e
==> [2021-08-11-14:49:43.444084] Creating stage lock resource-scorec-xjqqtbwx3wofw4lwwodf6vaehx5y5t7e
==> [2021-08-11-14:49:43.444577] Creating stage lock resource-mesquite-xjqqtbwx3wofw4lwwodf6vaehx5y5t7e
==> [2021-08-11-14:49:43.445821] Reading config file /projects/wind/spack-manager/spack/etc/spack/defaults/mirrors.yaml
==> [2021-08-11-14:49:43.448737] Checking existence of https://mirror.spack.io/_source-cache/git/trilinos/Trilinos.git/develop.tar.gz
==> [2021-08-11-14:49:43.449881] '/usr/bin/curl' '--stderr' '-' '-s' '-f' '-r' '0-0' 'https://mirror.spack.io/_source-cache/git/trilinos/Trilinos.git/develop.tar.gz'
==> [2021-08-11-14:49:44.165162] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/_source-cache/git/trilinos/Trilinos.git/develop.tar.gz
==> [2021-08-11-14:49:44.165659] Checking existence of https://mirror.spack.io/trilinos/trilinos-develop.tar.gz
==> [2021-08-11-14:49:44.167209] '/usr/bin/curl' '--stderr' '-' '-s' '-f' '-r' '0-0' 'https://mirror.spack.io/trilinos/trilinos-develop.tar.gz'
==> [2021-08-11-14:49:44.614361] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/trilinos/trilinos-develop.tar.gz
==> [2021-08-11-14:49:44.614893] Cloning git repository: https://github.com/trilinos/Trilinos.git on branch develop
==> [2021-08-11-14:49:44.617186] '/usr/bin/git' '--version'
==> [2021-08-11-14:49:44.626395] '/usr/bin/git' '-c' 'advice.detachedHead=false' '--version'
==> [2021-08-11-14:49:44.634428] '/usr/bin/git' '-c' 'advice.detachedHead=false' '--version'
==> [2021-08-11-14:49:44.641397] '/usr/bin/git' '-c' 'advice.detachedHead=false' 'clone' '--branch' 'develop' '--single-branch' '--depth' '1' 'https://github.com/trilinos/Trilinos.git'
Cloning into 'Trilinos'...
remote: Enumerating objects: 44939, done.
remote: Counting objects: 100% (44939/44939), done.
remote: Compressing objects: 100% (31639/31639), done.
remote: Total 44939 (delta 19978), reused 24625 (delta 12756), pack-reused 0
Receiving objects: 100% (44939/44939), 148.45 MiB | 12.30 MiB/s, done.
Resolving deltas: 100% (19978/19978), done.
Checking out files: 100% (51813/51813), done.
==> [2021-08-11-14:50:13.603800] Fetching from https://mirror.spack.io/_source-cache/git/trilinos/Trilinos.git/develop.tar.gz failed.
==> [2021-08-11-14:50:13.604170] Fetching from https://mirror.spack.io/trilinos/trilinos-develop.tar.gz failed.
==> [2021-08-11-14:53:18.515127] Checking existence of https://mirror.spack.io/_source-cache/git/ornl-cees/DataTransferKit.git/master.tar.gz
==> [2021-08-11-14:53:18.520477] '/usr/bin/curl' '--stderr' '-' '-s' '-f' '-r' '0-0' 'https://mirror.spack.io/_source-cache/git/ornl-cees/DataTransferKit.git/master.tar.gz'
==> [2021-08-11-14:53:19.110079] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/_source-cache/git/ornl-cees/DataTransferKit.git/master.tar.gz
==> [2021-08-11-14:53:19.110334] Checking existence of https://mirror.spack.io/trilinos/dtk-develop.tar.gz
==> [2021-08-11-14:53:19.111736] '/usr/bin/curl' '--stderr' '-' '-s' '-f' '-r' '0-0' 'https://mirror.spack.io/trilinos/dtk-develop.tar.gz'
==> [2021-08-11-14:53:19.663709] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/trilinos/dtk-develop.tar.gz
==> [2021-08-11-14:53:19.664254] Cloning git repository: https://github.com/ornl-cees/DataTransferKit.git on branch master
==> [2021-08-11-14:53:19.665772] '/usr/bin/git' '--version'
==> [2021-08-11-14:53:19.674938] '/usr/bin/git' '-c' 'advice.detachedHead=false' '--version'
==> [2021-08-11-14:53:19.684325] '/usr/bin/git' '-c' 'advice.detachedHead=false' '--version'
==> [2021-08-11-14:53:19.692226] '/usr/bin/git' '-c' 'advice.detachedHead=false' 'clone' '--branch' 'master' '--single-branch' '--depth' '1' 'https://github.com/ornl-cees/DataTransferKit.git'
Cloning into 'DataTransferKit'...
remote: Enumerating objects: 220, done.
remote: Counting objects: 100% (220/220), done.
remote: Compressing objects: 100% (210/210), done.
remote: Total 220 (delta 38), reused 63 (delta 2), pack-reused 0
Receiving objects: 100% (220/220), 243.14 KiB | 0 bytes/s, done.
Resolving deltas: 100% (38/38), done.
==> [2021-08-11-14:53:21.563262] '/usr/bin/git' '-c' 'advice.detachedHead=false' 'submodule' 'update' '--init' '--recursive'
Submodule 'cmake/tribits' (https://github.com/tribitspub/tribits) registered for path 'cmake/tribits'
Submodule 'data' (https://github.com/ornl-cees/DTKData.git) registered for path 'data'
Submodule 'arborx' (https://github.com/arborx/arborx.git) registered for path 'packages/Search'
Cloning into 'cmake/tribits'...
remote: Enumerating objects: 22577, done.
remote: Counting objects: 100% (2468/2468), done.
remote: Compressing objects: 100% (894/894), done.
remote: Total 22577 (delta 1427), reused 2347 (delta 1351), pack-reused 20109
Receiving objects: 100% (22577/22577), 16.54 MiB | 5.21 MiB/s, done.
Resolving deltas: 100% (16669/16669), done.
Submodule path 'cmake/tribits': checked out '06e00a308f96e0f79a367c3ffacc9a7f3d15a84e'
Cloning into 'data'...
remote: Enumerating objects: 43, done.
remote: Total 43 (delta 0), reused 0 (delta 0), pack-reused 43
Unpacking objects: 100% (43/43), done.
Submodule path 'data': checked out '496547228a0a7696569891c45ac04e0b8942b2a1'
Cloning into 'packages/Search'...
remote: Enumerating objects: 12500, done.
remote: Counting objects: 100% (718/718), done.
remote: Compressing objects: 100% (310/310), done.
remote: Total 12500 (delta 467), reused 576 (delta 407), pack-reused 11782
Receiving objects: 100% (12500/12500), 2.74 MiB | 2.70 MiB/s, done.
Resolving deltas: 100% (9142/9142), done.
Submodule path 'packages/Search': checked out 'f90641d8830571c1eb6a94231312580d6d2cc365'
==> [2021-08-11-14:53:32.291601] Fetching from https://mirror.spack.io/_source-cache/git/ornl-cees/DataTransferKit.git/master.tar.gz failed.
==> [2021-08-11-14:53:32.291955] Fetching from https://mirror.spack.io/trilinos/dtk-develop.tar.gz failed.
Traceback (most recent call last):
  File "/projects/wind/spack-manager/spack/bin/spack", line 77, in <module>
    sys.exit(spack.main.main())
  File "/projects/wind/spack-manager/spack/lib/spack/spack/main.py", line 771, in main
    return _invoke_command(command, parser, args, unknown)
  File "/projects/wind/spack-manager/spack/lib/spack/spack/main.py", line 499, in _invoke_command
    return_val = command(parser, args)
  File "/projects/wind/spack-manager/spack/lib/spack/spack/cmd/develop.py", line 100, in develop
    changed = env.develop(spec, path, clone)
  File "/projects/wind/spack-manager/spack/lib/spack/spack/environment.py", line 1166, in develop
    stage.steal_source(abspath)
  File "/projects/wind/spack-manager/spack/lib/spack/spack/util/pattern.py", line 24, in __call__
    for item in self.container]
  File "/projects/wind/spack-manager/spack/lib/spack/spack/util/pattern.py", line 24, in <listcomp>
    for item in self.container]
  File "/projects/wind/spack-manager/spack/lib/spack/spack/stage.py", line 531, in steal_source
    shutil.copytree(entry, d)
  File "/usr/lib64/python3.6/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/projects/wind/spack-manager/environments/cee/test/test2/trilinos/.git'

Information on your system

[psakiev@ascicgpu24 test]$ spack debug report
* **Spack:** 0.16.2-3052-729d66a
* **Python:** 3.6.8
* **Platform:** linux-rhel7-skylake_avx512
* **Concretizer:** original

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdev-specsAnything related to dev-specs in environmentsimpact-lowsnl-core-teamIssue for SNL Spack developers

    Type

    No type

    Projects

    Triaged bugs

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions