-
Notifications
You must be signed in to change notification settings - Fork 2.4k
consider using the pex library to allow spack users to configure where their python code resolves from #20504
Description
I don't know if this would have even been feasible before #17587 is merged, which introduces a much more natural and unified way to refer to python packages off of pypi.
I'm coming from using tools which very kindly allowed interchanging different sources of python repos: see pants configuration at https://www.pantsbuild.org/docs/python-third-party-dependencies#using-custom-repositories:
[python-repos]
indexes.add = ["https://custom-cheeseshop.net/simple"]Closer to the point, see pex, which is both a CLI tool and a python library: https://pex.readthedocs.io/en/latest/buildingpex.html#tailoring-requirement-resolution
In general, pex honors the same options as pip when it comes to resolving packages. Like pip, by default pex fetches artifacts from PyPI. This can be disabled with
--no-index.
If PyPI fetching is disabled, you will need to specify a search repository via-f/--find-links. This may be a directory on disk or a remote simple http server.
For example, you can delegate artifact fetching and resolution to pip wheel for whatever reason – perhaps you’re running a firewalled mirror – but continue to package with pex:
pip wheel -w /tmp/wheelhouse sphinx sphinx_rtd_theme
pex -f /tmp/wheelhouse --no-index -e sphinx:main -o sphinx.pex sphinx sphinx_rtd_themeDoes Spack currently have the ability to address any of these use cases? The reason I ask is because pex has this incredibly nice python library API for things that would normally involve manually scraping webpages:
from pex.resolver import download
download(['my-package==3.0'],
indexes=['https://my-corporate-pypi.repo'],
find_links=['/my/local/dir', 'https://my-corporate-flat.repo']) Pex also staunchly remains compatible with python 2.6 (pip install pex works).
Figuring out the appropriate way to pipe this sort of configuration into the user is definitely its own question, but I would be interested in knowing if it seems at all useful to look into how to depend on the pex library to expose this repo location choice via settings.