fill url, size and upload-time in Package.files, bump cache version#10677
Merged
radoering merged 4 commits intopython-poetry:mainfrom Jan 17, 2026
Merged
Conversation
Reviewer's GuideExtends Package.files metadata to include URL, size, and upload time from HTTP/JSON/legacy/PyPI sources and direct file origins while ensuring lock files still only store file and hash, with supporting fixtures/tests and a cache version bump. Sequence diagram for extended file metadata from PyPI to lock filesequenceDiagram
actor User
participant Installer
participant Provider
participant PyPiRepository
participant JsonLinkSource
participant HttpRepository
participant Package
participant Locker
User->>Installer: install package
Installer->>Provider: resolve dependency
Provider->>PyPiRepository: search package
PyPiRepository->>JsonLinkSource: request links
JsonLinkSource->>JsonLinkSource: parse JSON files
JsonLinkSource->>JsonLinkSource: create Link(size, upload_time)
JsonLinkSource-->>PyPiRepository: Link objects
PyPiRepository->>HttpRepository: get release info
HttpRepository->>HttpRepository: _links_to_data(links, data)
HttpRepository->>Package: set files[{file, hash, url, size, upload_time}]
PyPiRepository-->>Provider: Package with rich files
Provider-->>Installer: resolved Package
Installer->>Locker: lock project with Package
Locker->>Locker: _dump_package(package, target, env)
Locker->>Locker: sort and strip files to {file, hash}
Locker-->>Installer: lock data with minimal file metadata
Updated class diagram for package file metadata handlingclassDiagram
class Package {
+list~dict~ files
}
class HttpRepository {
+_links_to_data(links, data) dict
}
class PyPiRepository {
+_get_release_info(name, version) PackageInfo
}
class JsonLinkSource {
+_link_cache() LinkCache
}
class DirectOrigin {
+get_package_from_file(file_path) Package
+get_package_from_url(url) Package
}
class Provider {
+_search_for_file(dependency) Package
}
class Locker {
+_dump_package(package, target, env) dict
}
class CachedRepository {
+CACHE_VERSION Constraint
}
class Link {
+filename str
+url_without_fragment str
+size int
+upload_time_isoformat str
}
class PackageInfo {
+files list~dict~
}
HttpRepository --> PackageInfo : populates_files
HttpRepository --> Link : consumes
PyPiRepository --> PackageInfo : populates_files
JsonLinkSource --> Link : creates_with_size_upload_time
DirectOrigin --> Package : sets_files_with_size
Provider --> Package : no_direct_files_assignment
Locker --> Package : reads_files
Locker ..> Package : stores_only_file_and_hash
CachedRepository ..> PyPiRepository : used_for_caching
Flow diagram for package file metadata through repositories and lockerflowchart LR
A[PyPI JSON API] --> B[JsonLinkSource]
B -->|creates Link with size and upload_time| C[HttpRepository]
C -->|_links_to_data adds url, size, upload_time| D[Package.files]
A2[Legacy/PyPI file listing] --> C
E[DirectOrigin.get_package_from_file] -->|computes hash and size| D
D -->|read files| F[Locker._dump_package]
F -->|keep only file and hash| G[poetry.lock]
subgraph CRepo[CachedRepository]
C
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 3 issues, and left some high level feedback:
- The temporary dependency override in pyproject.toml pointing to the radoering/poetry-core git branch should be reverted back to the canonical python-poetry/poetry-core source (and version-pinned appropriately) before merging.
- The new legacy fixtures
get_legacy_dist_urlandget_legacy_dist_size_and_upload_timehave slightly misleading error messages (e.g. referencing.htmlin the JSON helper) and repeated path-discovery logic; consider aligning the messages with the actual file types and extracting the common package/version resolution into a shared helper to reduce duplication.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The temporary dependency override in pyproject.toml pointing to the radoering/poetry-core git branch should be reverted back to the canonical python-poetry/poetry-core source (and version-pinned appropriately) before merging.
- The new legacy fixtures `get_legacy_dist_url` and `get_legacy_dist_size_and_upload_time` have slightly misleading error messages (e.g. referencing `.html` in the JSON helper) and repeated path-discovery logic; consider aligning the messages with the actual file types and extracting the common package/version resolution into a shared helper to reduce duplication.
## Individual Comments
### Comment 1
<location> `src/poetry/repositories/http_repository.py:381-384` </location>
<code_context>
+ "url": link.url_without_fragment,
+ }
+ )
+ if link.size is not None:
+ files[-1]["size"] = link.size
+ if link.upload_time_isoformat is not None:
+ files[-1]["upload_time"] = link.upload_time_isoformat
if not files:
</code_context>
<issue_to_address>
**issue:** Accessing `files[-1]` can fail when `file_hash` is `None` and nothing is appended.
When `file_hash is None`, we only log and don’t append to `files`, but the subsequent size/upload_time blocks still run and access `files[-1]`. If the first (or any) link has no hash, `files` may be empty and this will raise `IndexError`. Either `continue` when `file_hash is None`, or only set size/upload_time when `files` is non-empty (e.g., by moving those assignments into the branch where the append occurs).
</issue_to_address>
### Comment 2
<location> `tests/repositories/fixtures/pypi.py:166-175` </location>
<code_context>
+def get_pypi_file_info(
</code_context>
<issue_to_address>
**suggestion:** Make get_pypi_file_info filename parsing robust to project names containing dashes
This logic will break for wheel filenames where the project name contains dashes (e.g. `my-package-1.0.0-py3-none-any.whl` yields `package_name == "my"`, `version == "package"`). That could make tests subtly wrong once such fixtures are added.
To make this helper robust, please use a safer parsing approach (e.g. `packaging.utils.parse_wheel_filename` / `packaging.tags`, or a more conservative `rsplit`-based pattern) so it continues to work for project names with dashes.
Suggested implementation:
```python
from collections.abc import Callable
from pathlib import Path
from packaging.utils import parse_wheel_filename
from requests import PreparedRequest
```
```python
@pytest.fixture
def get_pypi_file_info(
package_json_locations: list[Path],
) -> Callable[[str], dict[str, Any]]:
def get_file_info(name: str) -> dict[str, Any]:
if name.endswith(".whl"):
distribution, version, _, _ = parse_wheel_filename(name)
package_name = distribution
else:
package_name, version = name.removesuffix(".tar.gz").rsplit("-", 1)
path = package_json_locations[0] / package_name
if not path.exists():
raise RuntimeError(
```
</issue_to_address>
### Comment 3
<location> `tests/repositories/fixtures/legacy.py:267-270` </location>
<code_context>
+def get_legacy_dist_size_and_upload_time(
+ legacy_package_json_locations: list[Path],
+) -> Callable[[str], tuple[int | None, str | None]]:
+ def get_size_and_upload_time(name: str) -> tuple[int | None, str | None]:
+ package_name = name.split("-", 1)[0]
+ path = Path()
+ for location in legacy_package_json_locations:
+ path = location / f"{package_name}.json"
+ if path.exists():
+ break
+ if not path.exists():
+ raise RuntimeError(
+ f"Fixture for {package_name}.json not found in legacy fixtures"
+ )
</code_context>
<issue_to_address>
**nitpick (typo):** Fix misleading error message in legacy JSON fixture helper
The final `RuntimeError` still refers to an HTML URL, but this helper operates on `{package_name}.json` fixtures. Please update the message to mention the JSON fixture (`{package_name}.json`) and the missing file, rather than a URL, so test failures are clearer.
```suggestion
if not path.exists():
searched_paths = ", ".join(str(location) for location in legacy_package_json_locations)
raise RuntimeError(
f"Legacy JSON fixture file '{package_name}.json' not found in any of: {searched_paths}"
)
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
…a package index (python-poetry#10677) This makes it easier to get the required information to write a pylock.toml file. However, we have to make sure that it does not slip into poetry.lock (because we want to avoid unnecessary changes to the format.)
* url, size and upload-time of an artifact are cached now * JSON API is prefered to HTML API (size and upload-time are only available via JSON API)
radoering
force-pushed
the
repo-file-url-size-upload-time
branch
from
aa6647e to
f190371
Compare
radoering
added a commit
that referenced
this pull request
Jan 17, 2026
…a package index (#10677) This makes it easier to get the required information to write a pylock.toml file. However, we have to make sure that it does not slip into poetry.lock (because we want to avoid unnecessary changes to the format.)
radoering
added a commit
that referenced
this pull request
Jan 17, 2026
radoering
added a commit
that referenced
this pull request
Jan 17, 2026
* url, size and upload-time of an artifact are cached now * JSON API is prefered to HTML API (size and upload-time are only available via JSON API)
mwalbeck
pushed a commit
to mwalbeck/docker-python-poetry
that referenced
this pull request
Jan 19, 2026
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://github.com/python-poetry/poetry) ([changelog](https://python-poetry.org/history/)) | minor | ` 2.2.1` -> `2.3.0` | --- ### Release Notes <details> <summary>python-poetry/poetry (poetry)</summary> ### [`v2.3.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#230---2026-01-18) [Compare Source](python-poetry/poetry@2.2.1...2.3.0) ##### Added - **Add support for exporting `pylock.toml` files with `poetry-plugin-export`** ([#​10677](python-poetry/poetry#10677)). - Add support for specifying build constraints for dependencies ([#​10388](python-poetry/poetry#10388)). - Add support for publishing artifacts whose version is determined dynamically by the build-backend ([#​10644](python-poetry/poetry#10644)). - Add support for editable project plugins ([#​10661](python-poetry/poetry#10661)). - Check `requires-poetry` before any other validation ([#​10593](python-poetry/poetry#10593)). - Validate the content of `project.readme` when running `poetry check` ([#​10604](python-poetry/poetry#10604)). - Add the option to clear all caches by making the cache name in `poetry cache clear` optional ([#​10627](python-poetry/poetry#10627)). - Automatically update the cache for packages where the locked files differ from cached files ([#​10657](python-poetry/poetry#10657)). - Suggest to clear the cache if running a command with `--no-cache` solves an issue ([#​10585](python-poetry/poetry#10585)). - Propose `poetry init` when trying `poetry new` for an existing directory ([#​10563](python-poetry/poetry#10563)). - Add support for `poetry publish --skip-existing` for new Nexus OSS versions ([#​10603](python-poetry/poetry#10603)). - Show Poetry's own Python's path in `poetry debug info` ([#​10588](python-poetry/poetry#10588)). ##### Changed - **Drop support for Python 3.9** ([#​10634](python-poetry/poetry#10634)). - **Change the default of `installer.re-resolve` from `true` to `false`** ([#​10622](python-poetry/poetry#10622)). - **PEP 735 dependency groups are considered in the lock file hash** ([#​10621](python-poetry/poetry#10621)). - Deprecate `poetry.utils._compat.metadata`, which is sometimes used in plugins, in favor of `importlib.metadata` ([#​10634](python-poetry/poetry#10634)). - Improve managing free-threaded Python versions with `poetry python` ([#​10606](python-poetry/poetry#10606)). - Prefer JSON API to HTML API in legacy repositories ([#​10672](python-poetry/poetry#10672)). - When running `poetry init`, only add the readme field in the `pyproject.toml` if the readme file exists ([#​10679](python-poetry/poetry#10679)). - Raise an error if no hash can be determined for any distribution link of a package ([#​10673](python-poetry/poetry#10673)). - Require `dulwich>=0.25.0` ([#​10674](python-poetry/poetry#10674)). ##### Fixed - Fix an issue where `poetry remove` did not work for PEP 735 dependency groups with `include-group` items ([#​10587](python-poetry/poetry#10587)). - Fix an issue where `poetry remove` caused dangling `include-group` references in PEP 735 dependency groups ([#​10590](python-poetry/poetry#10590)). - Fix an issue where `poetry add` did not work for PEP 735 dependency groups with `include-group` items ([#​10636](python-poetry/poetry#10636)). - Fix an issue where PEP 735 dependency groups were not considered in the lock file hash ([#​10621](python-poetry/poetry#10621)). - Fix an issue where wrong markers were locked for a dependency that was required by several groups with different markers ([#​10613](python-poetry/poetry#10613)). - Fix an issue where non-deterministic markers were created in a method used by `poetry-plugin-export` ([#​10667](python-poetry/poetry#10667)). - Fix an issue where wrong wheels were chosen for installation in free-threaded Python environments if Poetry itself was not installed with free-threaded Python ([#​10614](python-poetry/poetry#10614)). - Fix an issue where `poetry publish` used the metadata of the project instead of the metadata of the build artifact ([#​10624](python-poetry/poetry#10624)). - Fix an issue where `poetry env use` just used another Python version instead of failing when the requested version was not supported by the project ([#​10685](python-poetry/poetry#10685)). - Fix an issue where `poetry env activate` returned the wrong command for `dash` ([#​10696](python-poetry/poetry#10696)). - Fix an issue where `data-dir` and `python.installation-dir` could not be set ([#​10595](python-poetry/poetry#10595)). - Fix an issue where Python and pip executables were not correctly detected on Windows ([#​10645](python-poetry/poetry#10645)). - Fix an issue where invalid template variables in `virtualenvs.prompt` caused an incomprehensible error message ([#​10648](python-poetry/poetry#10648)). ##### Docs - Add a warning about `~/.netrc` for Poetry credential configuration ([#​10630](python-poetry/poetry#10630)). - Clarify that the local configuration takes precedence over the global configuration ([#​10676](python-poetry/poetry#10676)). - Add an explanation in which cases `packages` are automatically detected ([#​10680](python-poetry/poetry#10680)). ##### poetry-core ([`2.3.0`](https://github.com/python-poetry/poetry-core/releases/tag/2.3.0)) - Normalize versions ([#​893](python-poetry/poetry-core#893)). - Fix an issue where unsatisfiable requirements did not raise an error ([#​891](python-poetry/poetry-core#891)). - Fix an issue where the implicit main group did not exist if it was explicitly declared as not having any dependencies ([#​892](python-poetry/poetry-core#892)). - Fix an issue where `python_full_version` markers with pre-release versions were parsed incorrectly ([#​893](python-poetry/poetry-core#893)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNDYuMCIsInVwZGF0ZWRJblZlciI6IjQxLjE0Ni4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/1654 Co-authored-by: renovate-bot <[email protected]> Co-committed-by: renovate-bot <[email protected]>
nothing-991
pushed a commit
to nothing-991/python-poetry
that referenced
this pull request
Feb 3, 2026
…a package index (python-poetry#10677) This makes it easier to get the required information to write a pylock.toml file. However, we have to make sure that it does not slip into poetry.lock (because we want to avoid unnecessary changes to the format.)
nothing-991
pushed a commit
to nothing-991/python-poetry
that referenced
this pull request
Feb 3, 2026
nothing-991
pushed a commit
to nothing-991/python-poetry
that referenced
this pull request
Feb 3, 2026
* url, size and upload-time of an artifact are cached now * JSON API is prefered to HTML API (size and upload-time are only available via JSON API)
nothing-991
pushed a commit
to nothing-991/python-poetry
that referenced
this pull request
Feb 3, 2026
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Check List
Actually, this is just an internal change that makes it easier to get the url, size and upload-time of a wheel/sdist. However, the change requires a bump of the cache version.
Requires: python-poetry/poetry-core#905
Related-to: python-poetry/poetry-plugin-export#336
Related-to: #10356
Related-to: #10646
Summary by Sourcery
Populate package file metadata with URLs, sizes, and upload times and ensure this additional information is not persisted into lock files, updating cache expectations accordingly.
New Features:
Enhancements:
Tests:
Chores: