Skip to content

Comments

Unify dependency iteration in ResolverState::get_dependencies#4515

Merged
konstin merged 1 commit intomainfrom
konsti/unify-dependency-iteration-in-get-dependencies
Jun 25, 2024
Merged

Unify dependency iteration in ResolverState::get_dependencies#4515
konstin merged 1 commit intomainfrom
konsti/unify-dependency-iteration-in-get-dependencies

Conversation

@konstin
Copy link
Member

@konstin konstin commented Jun 25, 2024

Upstack PR: #4430

Split out from #4430 according to #4430 (comment).

Comment on lines -958 to -965
let dependencies = match dependencies {
Ok(dependencies) => dependencies,
Err(err) => {
return Ok(Dependencies::Unavailable(
UnavailableVersion::ResolverError(uncapitalize(err.to_string())),
));
}
};
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@charliermarsh Do you know what this was about? It's only in this branch (in the non-root package branch we short-circuit on error) and the errors we're handling here look like they should be raised.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should they be raised? You don't think we should backtrack in these cases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as i can tell, the current implementation means that if there is a direct dependency (a root package dependency) where PubGrubSpecifier::try_from fails or which uses a banned URL, we mark those dependencies as unavailable, so the root package becomes unavailable(?). For regular dependencies, we already raise those errors directly using from_requirements(...)?.

This code looks like an artifact to me, but going through the git history unfortunately didn't clear this up either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there are some errors that we'd want to recover from though, like InvalidVersion or perhaps PubGrubSpecifierError?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think they are recoverable, as far as i can tell they lead to the root package being rejected in the next step, so you get the same error except it's wrapped in a message about the root package being unavailable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also do this only for the root package, which looks wrong.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are they rejected? Note that this does not return an error; it returns Ok(Dependencies::Unavailable).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is unreachable (both error cases are raised earlier), so i'm removing it

@konstin konstin force-pushed the konsti/unify-dependency-iteration-in-get-dependencies branch from 7eb4fdc to 384a290 Compare June 25, 2024 14:13
@konstin konstin changed the base branch from main to konsti/fork-before-transforming June 25, 2024 19:21
@konstin konstin changed the base branch from konsti/fork-before-transforming to main June 25, 2024 19:21
@konstin konstin merged commit ad42206 into main Jun 25, 2024
@konstin konstin deleted the konsti/unify-dependency-iteration-in-get-dependencies branch June 25, 2024 21:04
konstin added a commit that referenced this pull request Jun 25, 2024
Downstack PR: #4515 Upstack PR: #4481

Consider these two cases:

A:
```
werkzeug==2.0.0
werkzeug @ https://files.pythonhosted.org/packages/ff/1d/960bb4017c68674a1cb099534840f18d3def3ce44aed12b5ed8b78e0153e/Werkzeug-2.0.0-py3-none-any.whl
```

B:
```toml
dependencies = [
  "iniconfig == 1.1.1 ; python_version < '3.12'",
  "iniconfig @ git+https://github.com/pytest-dev/iniconfig@93f5930e668c0d1ddf4597e38dd0dea4e2665e7a ; python_version >= '3.12'",
]
```

In the first case, `werkzeug==2.0.0` should be overridden by the url. In
the second case `iniconfig == 1.1.1` is in a different fork and must
remain a registry distribution.

That means the conversion from `Requirement` to `PubGrubPackage` is
dependent on the other requirements of the package. We can either look
into the other packages immediately, or we can move the forking before
the conversion to `PubGrubDependencies` instead of after. Either version
requires a flat list of `Requirement`s to use. This refactoring gives us
this list.

I'll add support for both of the above cases in the forking urls branch
before merging this PR. I also have to move constraints over to this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants