6713 Introduce explicit package source by b-kamphorst · Pull Request #7658 · python-poetry/poetry

b-kamphorst · 2023-03-17T00:03:57Z

Pull Request Check List

Partially addresses #6713, specifically addresses #6879 (comment).

Added tests for changed code.
Updated documentation for changed code.

b-kamphorst · 2023-03-17T00:23:52Z

@radoering I prepared a PR for adding explicit sources as requested in #6879 (comment). The commit history is identical apart from the removal of supplemental-related changes in every commit. If you want I can clean up the commit history, but considering that we have been working on this already for quite a while and you may want to squash the changes anyway, I'll only make the effort if you explicitly (pun intended) request it.

src/poetry/factory.py

dimbleby · 2023-03-18T14:27:55Z

if default is deprecated, what is the intended replacement for that? ie how do I now disable use of pypi and insist that everything come from my private repository?

primary doesn't seem to do it, or anyway the docs only say that primary sources are considered before the default source: they don't give me any way to remove that default source. (It's possible that this is a documentation gap or even a reading comprehension gap).

I've said this a couple of times in related MRs and issues but to avoid such gaps can I once again encourage development of a list or table of: some common use cases for multiple repositories, and how those use cases are achieved by any new proposal.

I think #6713 (comment) was my first try at asking for this, and should provide a starting point.

radoering · 2023-03-18T15:02:38Z

if default is deprecated, what is the intended replacement for that? ie how do I now disable use of pypi and insist that everything come from my private repository?

I've not reviewed this PR yet, but this PR should only introduce a new source type called explicit and instead of having a flag for each type like

secondary=false
default=false
explicit=true

in pyproject.toml, just define one property, e.g. priority="default" or priority="explicit".

Deprecating default sources is not in scope of this PR. #7430 is one approach. I have another approach in mind, which allows to set the priority of PyPI and thus covers more use cases. Then, you can disable it by setting its priority to "explicit". IMO default is so broken it is difficult to fix and it's not really required as a priority.

radoering

Some first feedback.

docs/dependency-specification.md

docs/repositories.md

src/poetry/console/commands/source/add.py

src/poetry/repositories/repository_pool.py

src/poetry/json/schemas/poetry.json

docs/cli.md

docs/repositories.md

src/poetry/factory.py

src/poetry/json/schemas/poetry.json

src/poetry/console/commands/source/add.py

b-kamphorst · 2023-04-07T17:13:16Z

I am quite sure that I addressed all feedback, but GitHub seems to think I have one open requested change (conversation view). In files view it states that everything is resolved. Either way: if there are remaining changes please let me know so that we can wrap this up :)

radoering · 2023-04-08T11:05:40Z

I squashed the changes into two commits and added some minor changes:

added some tests for the last bit of coverage and fixed a special case for updating an existing source via source add
reverted removal of has_primary_repositories() (Might be used by plugins. I wondered if we should deprecate it, but decided against it for now.)
added a docstring to RepositoryPool.repositories and another property RepositoryPool.all_repositories. I wondered if repositories should actually return all repositories (including explicit ones), but decided against it (see docstring for reasoning).

@b-kamphorst Please let me know if the last two points make sense to you?

github-actions · 2023-04-08T11:07:57Z

Deploy preview for website ready!

✅ Preview
https://website-oawgrkgna-python-poetry.vercel.app

Built with commit 82ae9f3.
This pull request is being automatically deployed with vercel-action

b-kamphorst · 2023-04-10T18:10:36Z

reverted removal of has_primary_repositories() (Might be used by plugins. I wondered if we should deprecate it, but decided against it for now.)

It might indeed be, so reverting the removal is completely justified. Perhaps deprecation is something to reconsider when you decided on the future of "default" repositories, specifically if that also deprecates RepositoryPool.has_default.

added a docstring to RepositoryPool.repositories and another property RepositoryPool.all_repositories. I wondered if repositories should actually return all repositories (including explicit ones), but decided against it (see docstring for reasoning).

I think this makes sense. If we could design this from scratch we'd probably opt to rename RepositoryPool.repositories to something that signals it returns a selection / applies a filter, but we can't. RepositoryPool.repositories_unfiltered is longer but perhaps repositories/repositories_unfiltered is slightly more informative than repositories/all_repositories? I don't have strong feelings about this though.

…fault" and "secondary", adjust cli accordingly (#7658) Co-authored-by: Randy Döring <[email protected]>

Explicit sources are considered only for packages that explicitly indicate their source. Co-authored-by: Randy Döring <[email protected]>

radoering

@b-kamphorst Thanks for your feedback and your patience. ❤️

RepositoryPool.repositories_unfiltered is longer but perhaps repositories/repositories_unfiltered is slightly more informative than repositories/all_repositories? I don't have strong feelings about this though.

For the record: I'll stick to all_repositories here.

…fault" and "secondary", adjust cli accordingly (#7658) Co-authored-by: Randy Döring <[email protected]>

Explicit sources are considered only for packages that explicitly indicate their source. Co-authored-by: Randy Döring <[email protected]>

simon-liebehenschel · 2023-05-23T10:14:51Z

I updated to 1.5.0, and I was trying to test this changes. I expected that I can just replace

default = false
secondary = true

with

priority = "explicit"

for every [[tool.poetry.source]] which point to a private package registry.

As it turns out, this does not work. Poetry can find packages with the old pyproject.toml, but can not find packages after the mentioned replacement.
More detailed example: https://github.com/orgs/python-poetry/discussions/7988

Am I missing something? Did I overlook something in the migration instructions?

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://python-poetry.org/) ([source](https://github.com/python-poetry/poetry), [changelog](https://python-poetry.org/history/)) | minor | `1.4.2` -> `1.5.0` | --- ### Release Notes <details> <summary>python-poetry/poetry</summary> ### [`v1.5.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#150---2023-05-19) [Compare Source](python-poetry/poetry@1.4.2...1.5.0) ##### Added - **Introduce the new source priorities `explicit` and `supplemental`** ([#7658](python-poetry/poetry#7658), [#6879](python-poetry/poetry#6879)). - **Introduce the option to configure the priority of the implicit PyPI source** ([#7801](python-poetry/poetry#7801)). - Add handling for corrupt cache files ([#7453](python-poetry/poetry#7453)). - Improve caching of URL and git dependencies ([#7693](python-poetry/poetry#7693), [#7473](python-poetry/poetry#7473)). - Add option to skip installing directory dependencies ([#6845](python-poetry/poetry#6845), [#7923](python-poetry/poetry#7923)). - Add `--executable` option to `poetry env info` ([#7547](python-poetry/poetry#7547)). - Add `--top-level` option to `poetry show` ([#7415](python-poetry/poetry#7415)). - Add `--lock` option to `poetry remove` ([#7917](python-poetry/poetry#7917)). - Add experimental `POETRY_REQUESTS_TIMEOUT` option ([#7081](python-poetry/poetry#7081)). - Improve performance of wheel inspection by avoiding unnecessary file copy operations ([#7916](python-poetry/poetry#7916)). ##### Changed - **Remove the old deprecated installer and the corresponding setting `experimental.new-installer`** ([#7356](python-poetry/poetry#7356)). - **Introduce `priority` key for sources and deprecate flags `default` and `secondary`** ([#7658](python-poetry/poetry#7658)). - Deprecate `poetry run <entry point>` if the entry point was not previously installed via `poetry install` ([#7606](python-poetry/poetry#7606)). - Only write the lock file if the installation succeeds ([#7498](python-poetry/poetry#7498)). - Do not write the unused package category into the lock file ([#7637](python-poetry/poetry#7637)). ##### Fixed - Fix an issue where Poetry's internal pyproject.toml continually grows larger with empty lines ([#7705](python-poetry/poetry#7705)). - Fix an issue where Poetry crashes due to corrupt cache files ([#7453](python-poetry/poetry#7453)). - Fix an issue where the `Retry-After` in HTTP responses was not respected and retries were handled inconsistently ([#7072](python-poetry/poetry#7072)). - Fix an issue where Poetry silently ignored invalid groups ([#7529](python-poetry/poetry#7529)). - Fix an issue where Poetry does not find a compatible Python version if not given explicitly ([#7771](python-poetry/poetry#7771)). - Fix an issue where the `direct_url.json` of an editable install from a git dependency was invalid ([#7473](python-poetry/poetry#7473)). - Fix an issue where error messages from build backends were not decoded correctly ([#7781](python-poetry/poetry#7781)). - Fix an infinite loop when adding certain dependencies ([#7405](python-poetry/poetry#7405)). - Fix an issue where pre-commit hooks skip pyproject.toml files in subdirectories ([#7239](python-poetry/poetry#7239)). - Fix an issue where pre-commit hooks do not use the expected Python version ([#6989](python-poetry/poetry#6989)). - Fix an issue where an unclear error message is printed if the project name is the same as one of its dependencies ([#7757](python-poetry/poetry#7757)). - Fix an issue where `poetry install` returns a zero exit status even though the build script failed ([#7812](python-poetry/poetry#7812)). - Fix an issue where an existing `.venv` was not used if `in-project` was not set ([#7792](python-poetry/poetry#7792)). - Fix an issue where multiple extras passed to `poetry add` were not parsed correctly ([#7836](python-poetry/poetry#7836)). - Fix an issue where `poetry shell` did not send a newline to `fish` ([#7884](python-poetry/poetry#7884)). - Fix an issue where `poetry update --lock` printed operations that were not executed ([#7915](python-poetry/poetry#7915)). - Fix an issue where `poetry add --lock` did perform a full update of all dependencies ([#7920](python-poetry/poetry#7920)). - Fix an issue where `poetry shell` did not work with `nushell` ([#7919](python-poetry/poetry#7919)). - Fix an issue where subprocess calls failed on Python 3.7 ([#7932](python-poetry/poetry#7932)). - Fix an issue where keyring was called even though the password was stored in an environment variable ([#7928](python-poetry/poetry#7928)). ##### Docs - Add information about what to use instead of `--dev` ([#7647](python-poetry/poetry#7647)). - Promote semantic versioning less aggressively ([#7517](python-poetry/poetry#7517)). - Explain Poetry's own versioning scheme in the FAQ ([#7517](python-poetry/poetry#7517)). - Update documentation for configuration with environment variables ([#6711](python-poetry/poetry#6711)). - Add details how to disable the virtualenv prompt ([#7874](python-poetry/poetry#7874)). - Improve documentation on whether to commit `poetry.lock` ([#7506](python-poetry/poetry#7506)). - Improve documentation of `virtualenv.create` ([#7608](python-poetry/poetry#7608)). ##### poetry-core ([`1.6.0`](https://github.com/python-poetry/poetry-core/releases/tag/1.6.0)) - Improve error message for invalid markers ([#569](python-poetry/poetry-core#569)). - Increase robustness when deleting temporary directories on Windows ([#460](python-poetry/poetry-core#460)). - Replace `tomlkit` with `tomli`, which changes the interface of some *internal* classes ([#483](python-poetry/poetry-core#483)). - Deprecate `Package.category` ([#561](python-poetry/poetry-core#561)). - Fix a performance regression in marker handling ([#568](python-poetry/poetry-core#568)). - Fix an issue where wildcard version constraints were not handled correctly ([#402](python-poetry/poetry-core#402)). - Fix an issue where `poetry build` created duplicate Python classifiers if they were specified manually ([#578](python-poetry/poetry-core#578)). - Fix an issue where local versions where not handled correctly ([#579](python-poetry/poetry-core#579)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/717 Co-authored-by: renovate-bot <[email protected]> Co-committed-by: renovate-bot <[email protected]>

github-actions · 2024-03-03T21:05:29Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

b-kamphorst marked this pull request as ready for review March 17, 2023 00:23

b-kamphorst commented Mar 17, 2023

View reviewed changes

src/poetry/factory.py Outdated Show resolved Hide resolved

b-kamphorst marked this pull request as draft March 17, 2023 07:48

radoering reviewed Mar 18, 2023

View reviewed changes

b-kamphorst requested a review from radoering March 19, 2023 21:29

b-kamphorst marked this pull request as ready for review March 19, 2023 21:30

TCherici reviewed Mar 20, 2023

View reviewed changes

src/poetry/json/schemas/poetry.json Outdated Show resolved Hide resolved

radoering requested changes Apr 2, 2023

View reviewed changes

radoering added impact/changelog Requires a changelog entry impact/docs Contains or requires documentation changes area/sources Releated to package sources/indexes/repositories labels Apr 8, 2023

b-kamphorst and others added 3 commits April 15, 2023 18:35

sources: introduce "priority" key for sources and deprecate flags "de…

3ed0ede

…fault" and "secondary", adjust cli accordingly (#7658) Co-authored-by: Randy Döring <[email protected]>

sources: introduce priority "explicit" (#7658)

1ed2b83

Explicit sources are considered only for packages that explicitly indicate their source. Co-authored-by: Randy Döring <[email protected]>

tests: consistently use fixture_dir

82ae9f3

radoering approved these changes Apr 16, 2023

View reviewed changes

radoering merged commit 3855bc5 into python-poetry:master Apr 16, 2023

radoering added a commit that referenced this pull request Apr 16, 2023

sources: introduce "priority" key for sources and deprecate flags "de…

5806b42

…fault" and "secondary", adjust cli accordingly (#7658) Co-authored-by: Randy Döring <[email protected]>

radoering added a commit that referenced this pull request Apr 16, 2023

sources: introduce priority "explicit" (#7658)

4c37735

Explicit sources are considered only for packages that explicitly indicate their source. Co-authored-by: Randy Döring <[email protected]>

radoering mentioned this pull request Apr 16, 2023

Replace the secondary source type with more granular types #6713

Closed

b-kamphorst deleted the 5959-introduce-explicit-package-source branch April 16, 2023 19:57

Secrus mentioned this pull request Apr 17, 2023

6713 Introduce supplemental package source #6879

Merged

2 tasks

dimbleby mentioned this pull request Apr 19, 2023

problem of resolving nested dependencies with private repositories #7813

Closed

4 tasks

radoering mentioned this pull request Apr 30, 2023

1.5.0 Release #7853

Closed

ralbertazzi mentioned this pull request May 5, 2023

Instructions for installing PyTorch #6409

Open

1 task

radoering mentioned this pull request May 18, 2023

release: bump version to 1.5.0 #7947

Merged

celsiusnarhwal mentioned this pull request May 23, 2023

Fails with poetry-core 1.6.0 celsiusnarhwal/poetry-source-env#1

Closed

b-kamphorst mentioned this pull request May 23, 2023

7988 fix resolving subdependencies from explicit source #7992

Closed

2 tasks

haffi96 mentioned this pull request Aug 3, 2023

fix: update rabbitmq config bink-archive/vela#196

Merged

jeffwidman mentioned this pull request Aug 18, 2023

need to update to match poetry's new sources dependabot/dependabot-core#7839

Closed

github-actions bot locked as resolved and limited conversation to collaborators Mar 3, 2024

Comments

Conversation

b-kamphorst commented Mar 17, 2023

Pull Request Check List

Uh oh!

b-kamphorst commented Mar 17, 2023

Uh oh!

Uh oh!

dimbleby commented Mar 18, 2023

Uh oh!

radoering commented Mar 18, 2023

Uh oh!

radoering left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

b-kamphorst commented Apr 7, 2023

Uh oh!

radoering commented Apr 8, 2023

Uh oh!

github-actions bot commented Apr 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

b-kamphorst commented Apr 10, 2023

Uh oh!

radoering left a comment

Choose a reason for hiding this comment

Uh oh!

simon-liebehenschel commented May 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions bot commented Apr 8, 2023 •

edited

Loading

simon-liebehenschel commented May 23, 2023 •

edited

Loading