Skip to content

Poetry not respecting order of evaluation for repository sources #2339

@TheFriendlyCoder

Description

@TheFriendlyCoder
  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • OS version and name: Mac OS 10.15.3

  • Poetry version: 1.0.5

Issue

So, I am trying to configure a pyproject.toml file that supports pulling files from pypi.org whenever possible (ie: because on my site the performance is better) but will support pulling packages that aren't found on pypi.org from a secondary location (ie: a private pypi repo). So to start with I added a section like what follows to a sample toml file for testing:

[[tool.poetry.source]]
name = 'default'
url = 'https://pypi.python.org/simple'
default = true

[[tool.poetry.source]]
name = "private"
url = "https://private/server/url
secondary = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Based on the docs, I was under the impression that by putting default=true on the first repo config, and secondary=true on the other one, that my goal would be achieved: if a package exists on pypi.org it'd pull it from there and if not it'd pull it from my secondary repo. However, that does not seem to be the case.

For the sake of this discussion I am using average performance metrics from running poetry lock on this toml file and comparing the length of time it takes to resolve the dependencies of the one Python package mentioned (ie: sphinx which is a standard Python package available on pypi.org).

So, running poetry lock on this file using the configuration I posted above takes about 20-25 seconds to complete. For comparison purposes I simply removed the second source definition from the toml file giving me:

[[tool.poetry.source]]
name = 'default'
url = 'https://pypi.python.org/simple'
default = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Re-running the same common against this toml file takes between 2 and 3 seconds to complete - a full order of magnitude difference in performance. Now, for my 3rd and final test I removed the first repo source leaving just the second one, giving me the following configuration block:

[[tool.poetry.source]]
name = "private"
url = "https://private/server/url
secondary = true

[tool.poetry.dependencies]
python = "^3.6"
sphinx = "*"

Re-running the same command again takes about the same 20-25 seconds to complete (NOTE: our private pypi repository is also a mirror of the public pypi.org repo so I can compare the same package versions and resolution times in all cases).

So, based on these results, the only explanation I can see is that when the second repository definition is in place the public pypi.org repo is being ignored or something and the secondary / private repo is still being accessed for some reason.

My expectation here is that poetry would iterate over the various repos defined in the toml file in order of declaration. For each package listed in the toml file it should then try to access / index each package against each repository stopping when it finds the first match. If this were true then I would have expected my first test case above to perform identically to the second one because the package I'm testing (sphinx) should exist on pypi.org, and all of it's transitive dependencies will be available there as well. However this is obviously not the case.

So my question is - is this a bug or is there some additional configuration options I need to specify to get this to work as I was expecting?

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/repoMeta-issues for the repository/forge itselfkind/bugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions