new packages (py-metaphlan, py-hclust2, iqtree2) + updates to others (py-dendropy, py-phylophlan, py-pkgconfig) by luke-dt · Pull Request #32936 · spack/spack

luke-dt · 2022-09-30T23:07:26Z

I added a metaphlan package, and in the process cleaned up the py-phylophlan package with the dependencies from the bioconda recipe, added some newer versions of dependencies, and fixed py-pkgconfig which was throwing an error while building

spackbot-app · 2022-09-30T23:07:34Z

Hi @luke-dt! I noticed that the following package(s) don't yet have maintainers:

iqtree2
py-hclust2
py-metaphlan

Are you interested in adopting any of these package(s)? If so, simply add the following to the package class:

    maintainers = ['luke-dt']

If not, could you contact the developers of this package and see if they are interested? You can quickly see who has worked on a package with spack blame:

$ spack blame iqtree2

Thank you for your help! Please don't add maintainers without their consent.

You don't have to be a Spack expert or package developer in order to be a "maintainer," it just gives us a list of users willing to review PRs or debug issues relating to this package. A package can have multiple maintainers; just add a list of GitHub handles of anyone who wants to volunteer.

var/spack/repos/builtin/packages/iqtree2/package.py

var/spack/repos/builtin/packages/py-dendropy/package.py

var/spack/repos/builtin/packages/py-metaphlan/package.py

var/spack/repos/builtin/packages/py-phylophlan/package.py

adamjstewart · 2022-10-01T02:36:32Z

var/spack/repos/builtin/packages/py-pkgconfig/package.py

-    depends_on("py-poetry-core@1:", when="@1.5.5:", type="build")
-    depends_on("py-poetry-core@0.12:", when="@1.5:", type="build")
+    depends_on("py-poetry@1:", when="@1.5.2:", type="build")
+    depends_on("[email protected]:", when="@1.5:", type="build")


This is incorrect

I kept getting "No module named poetry" error while installing this, but I'll test it again without this

@adamjstewart I'm having the same issue when re-building. I think it's a similar problem to #29308 Here is the build log:

==> py-pkgconfig: Executing phase: 'install' ==> [2022-10-01-10:45:23.400953] '/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/python-3.9.13-5zn3hui4ysxknqhfvmbsupmf5ysefebn/bin/python3.9' '-m' 'pip' '-vvv' '--no-input' '--no-cache-dir' '--disable-pip-version-check' 'install' '--no-deps' '--ignore-installed' '--no-build-isolation' '--no-warn-script-location' '--no-index' '--prefix=/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pkgconfig-1.5.5-yzezoqcuwxhqkkc4z6o7purymq3ufhe6' '.'i.org/simple Created temporary directory: /tmp/pip-ephem-wheel-cache-v265qa90 Created temporary directory: /tmp/pip-build-tracker-rrrnkivo Initialized build tracking at /tmp/pip-build-tracker-rrrnkivo Created build tracker: /tmp/pip-build-tracker-rrrnkivo Entered build tracker: /tmp/pip-build-tracker-rrrnkivo Created temporary directory: /tmp/pip-install-jbh6ib2n Processing /home/lukedt/Documents/GitHub/spack/var/spack/stage/spack-stage-py-pkgconfig-1.5.5-yzezoqcuwxhqkkc4z6o7purymq3ufhe6/spack-src Added file:///home/lukedt/Documents/GitHub/spack/var/spack/stage/spack-stage-py-pkgconfig-1.5.5-yzezoqcuwxhqkkc4z6o7purymq3ufhe6/spack-src to build tracker '/tmp/pip-build-tracker-rrrnkivo' Created temporary directory: /tmp/pip-modern-metadata-ge9ca6vp Preparing metadata (pyproject.toml): started Running command Preparing metadata (pyproject.toml) Preparing metadata (pyproject.toml): finished with status 'done' ERROR: Exception: Traceback (most recent call last): File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper status = run_func(*args) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper return func(self, options, args) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 341, in run requirement_set = resolver.resolve( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve collected = self.factory.collect_root_requirements(root_reqs) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements req = self._make_requirement_from_install_req( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req cand = self._make_candidate_from_link( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__ super().__init__( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__ self.dist = self._prepare() File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare dist = self._prepare_distribution() File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 428, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 497, in _prepare_linked_requirement dist = _get_prepared_distribution( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata self.req.prepare_metadata() File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 526, in prepare_metadata self.metadata_directory = generate_metadata( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_internal/utils/misc.py", line 706, in prepare_metadata_for_build_wheel return super().prepare_metadata_for_build_wheel( File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 188, in prepare_metadata_for_build_wheel return self._call_hook('prepare_metadata_for_build_wheel', { File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last): File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-pip-22.1.2-frngqanail6q44luijakcnmb3iox3fws/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend obj = import_module(mod_path) File "/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/python-3.9.13-5zn3hui4ysxknqhfvmbsupmf5ysefebn/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked ModuleNotFoundError: No module named 'poetry' Removed file:///home/lukedt/Documents/GitHub/spack/var/spack/stage/spack-stage-py-pkgconfig-1.5.5-yzezoqcuwxhqkkc4z6o7purymq3ufhe6/spack-src from build tracker '/tmp/pip-build-tracker-rrrnkivo' Removed build tracker: '/tmp/pip-build-tracker-rrrnkivo'

Can you share spack-built-env.txt and also the output of python3 -m sysconfig?

Are you using Spack-installed Python or the system Python as an external package?

Okay, so you're using Spack-installed Python. So it's not an issue with sysconfig/distutils hacking.

Another possibility is that this is related to #32375. Can you run:

$ tree /home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-poetry-core-1.2.0-xr4smvmirljhej6hnwtnsb5bmlrnyg6b/lib/python3.9/site-packages

(or find if tree is not installed)

I'm guessing that the entire poetry directory and all of its contents will be missing. Do you have a global .gitignore somewhere? Can you upload that?

/home/lukedt/Documents/GitHub/spack/opt/spack/linux-fedora36-zen/gcc-8.5.0/py-poetry-core-1.2.0-xr4smvmirljhej6hnwtnsb5bmlrnyg6b/lib/python3.9/site-packages └── poetry_core-1.2.0.dist-info ├── direct_url.json ├── INSTALLER ├── LICENSE ├── METADATA ├── RECORD ├── REQUESTED └── WHEEL 1 directory, 7 files

I don't have a global .gitignore, just the one in ~/Documents/GitHub/spack

Must be pulling it in from somewhere else then. I'll see if I can hack together a solution since you're the 3rd person who's reported this issue.

thanks for the assistance!

so oddly enough, my latest commit (e64fd01) builds OK on my machine without this error...

…pack#32929)

Co-authored-by: Annop Wongwathanarat <[email protected]>

* re2c depends on cmake on Windows * Winbison properly added to bootstrap package search list

)

* bring in changes from axom repo Co-authored-by: white238 <[email protected]> Co-authored-by: Tamara Dahlgren <[email protected]>

…tall --source` (spack#32953)

adamjstewart · 2022-10-04T16:18:57Z

Looks like a bad rebase...

luke-dt · 2022-10-04T16:31:39Z

Looks like a bad rebase...

Yeah, I was trying to fetch the upstream changes and test on a different machine and messed that up.

…pkgconfig. Poetry install still broken

var/spack/repos/builtin/packages/iqtree2/package.py

var/spack/repos/builtin/packages/py-dendropy/package.py

var/spack/repos/builtin/packages/py-pkgconfig/package.py

…deps

adamjstewart · 2022-10-07T02:08:20Z

@spackbot run pipeline

spackbot-app · 2022-10-07T02:08:24Z

I've started that pipeline for you!

luke-dt · 2022-10-08T14:27:36Z

What check is it failing? I don't really understand what the pipeline is testing.

adamjstewart · 2022-10-08T16:31:04Z

This is a false positive. I'll try rebasing to see if that solves the issue. If not, I'll ping some of our CI folks.

adamjstewart · 2022-10-08T16:31:31Z

It looks like I don't have permission to rebase. Can you either rebase or merge from develop to update your branch?

luke-dt added 4 commits September 28, 2022 11:29

added metaphlan v4, cleaned up phylophlan

741c406

added iqtree2

c26d870

fixed phylophlan, builds now

87212e4

changed config.yaml to default

e864334

spackbot-app bot added dependencies new-package new-version python update-package labels Sep 30, 2022

spackbot-app bot requested review from adamjstewart and pradyunsg September 30, 2022 23:07

luke-dt added 2 commits September 30, 2022 18:13

fixed style

abbfb65

Merge branch 'develop' into package/py-metaphlan

e9318c2

adamjstewart self-assigned this Oct 1, 2022

adamjstewart requested changes Oct 1, 2022

View reviewed changes

This was referenced Oct 2, 2022

Installation issue: py-pkgconfig needs py-poetry, not py-poetry-core? #29308

Closed

Poetry-core picks up unrelated .gitignore which results in an empty wheel python-poetry/poetry#5547

Closed

manuelakuhn and others added 12 commits October 4, 2022 08:24

py-jsonschema: add 4.16.0 and new package py-hatch-fancy-pypi-readme (s…

cfb5ecf

…pack#32929)

acfl: add v22.1 (spack#32915)

67b8f77

Co-authored-by: Annop Wongwathanarat <[email protected]>

Fixup errors introduced by Clingo Pr: (spack#32905)

52125f1

* re2c depends on cmake on Windows * Winbison properly added to bootstrap package search list

Set CMAKE_HIP_ARCHITECTURES with the value of amdgpu_target (spack#32901

af88971

)

libtiff: default to +zlib+jpeg (spack#32945)

76baf2f

octave: add version 7.2.0 (spack#32943)

06342a9

simgrid new releases (spack#32920)

c515ded

[rocksdb] Added rtti variant (spack#32918)

5a28537

rvs binary path updated for 5.2 rocm release (spack#32892)

4130f9d

Add checksum for py-pytest-runner 6.0.0 (spack#32957)

74dcf86

py-einops: add v0.5.0 (spack#32959)

4dd90d2

Replace repo with the NVIDIA one (spack#32951)

69e8ce0

white238 and others added 7 commits October 4, 2022 08:24

Axom: bring in changes from axom repo (spack#32643)

7efb9ea

* bring in changes from axom repo Co-authored-by: white238 <[email protected]> Co-authored-by: Tamara Dahlgren <[email protected]>

Add checksum for py-pyparsing 3.0.9 (spack#32952)

7cc2b62

rdma-core: fix syntax for external discoverability (spack#32962)

6746547

Add checksum for py-flatbuffers 2.0.7 (spack#32955)

80de1c0

amrex: add v22.10 (spack#32966)

d82b01e

Remove CMakePackage.define alias from most packages (spack#32950)

1821bc0

Bug fix for ca-certificates-mozilla/package.py to enable `spack ins…

e62aeae

…tall --source` (spack#32953)

spackbot-app bot added conflicts core PR affects Spack core functionality maintainers new-variant labels Oct 4, 2022

luke-dt added 3 commits October 6, 2022 13:21

Merge branch 'develop' into package/py-metaphlan

8396b4a

made suggested changes to iqtree2, py-dendropy, py-metaphlan, and py-…

fc64ebd

…pkgconfig. Poetry install still broken

reverted py-pkgconfig deps to poetry-core

a75ff06

adamjstewart requested changes Oct 6, 2022

View reviewed changes

var/spack/repos/builtin/packages/iqtree2/package.py Outdated Show resolved Hide resolved

var/spack/repos/builtin/packages/py-dendropy/package.py Outdated Show resolved Hide resolved

var/spack/repos/builtin/packages/py-pkgconfig/package.py Outdated Show resolved Hide resolved

made iqtree2 less dedundant, changes to py-dendropy and py-pkgconfig …

e64fd01

…deps

adamjstewart approved these changes Oct 7, 2022

View reviewed changes

Merge branch 'develop' into package/py-metaphlan

c5490f4

adamjstewart enabled auto-merge (squash) October 8, 2022 16:47

adamjstewart merged commit 6c2df00 into spack:develop Oct 8, 2022

luke-dt deleted the package/py-metaphlan branch October 8, 2022 23:15

adamjstewart mentioned this pull request Oct 11, 2022

py-poetry-core: jail git to stage directory #33181

Merged

adamjstewart mentioned this pull request Oct 18, 2023

Installation issue: py-pkgconfig #40601

Closed

4 tasks

Conversation

luke-dt commented Sep 30, 2022

Uh oh!

spackbot-app bot commented Sep 30, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamjstewart commented Oct 4, 2022

Uh oh!

luke-dt commented Oct 4, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adamjstewart commented Oct 7, 2022

Uh oh!

spackbot-app bot commented Oct 7, 2022

Uh oh!

luke-dt commented Oct 8, 2022

Uh oh!

adamjstewart commented Oct 8, 2022

Uh oh!

adamjstewart commented Oct 8, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants