cache: automatic refresh if files of locked package differ from files in cache by radoering · Pull Request #10657 · python-poetry/poetry

radoering · 2025-12-11T17:27:50Z

When working on a project with many developers, unnecessary diffs in the lockfile due to outdated caches of some developers are an annoying issue. Outdated caches are normally the result of postponed artifact uploads. This change detects such diffs and refreshes the cache automatically.

Pull Request Check List

Added tests for changed code.
Updated documentation for changed code.

Summary by Sourcery

Automatically refresh cached package metadata when lockfile files differ from repository cache to avoid unnecessary lockfile diffs.

New Features:

Add repository pool refresh mechanism that invalidates cached package metadata and reloads it from the underlying repository when needed.

Bug Fixes:

Ensure packages completed from the pool are refreshed when their cached file list is outdated compared to the locked package, preventing spurious lockfile changes.

Enhancements:

Track refreshed packages in the provider to avoid repeated cache refreshes for the same package version and source.
Expose a forget operation on cached repositories to clear cached release info entries by name and version.

Tests:

Add provider tests covering cache refresh behavior when package files in the lock differ from cached repository data.
Extend lock and add command tests to account for cache refresh logic and ensure no unnecessary updates occur.
Add cached repository tests verifying release info caching, cache disabling, and behavior after clearing cached entries.

sourcery-ai · 2025-12-11T17:27:57Z

Reviewer's Guide

Adds automatic cache refresh for locked packages when their file lists differ from cached repository data, plus supporting repository APIs and tests to ensure outdated caches are corrected and not refreshed unnecessarily.

Sequence diagram for automatic cache refresh on package file mismatch

sequenceDiagram
    participant Provider
    participant RepositoryPool
    participant CachedRepository
    participant ReleaseCache as Release_cache

    Provider->>RepositoryPool: package(pretty_name, version, repository_name)
    RepositoryPool->>CachedRepository: package(name, version)
    CachedRepository->>ReleaseCache: get_release_info(canonical_name, version)
    ReleaseCache-->>CachedRepository: release_info
    CachedRepository-->>RepositoryPool: Package(pool_package)
    RepositoryPool-->>Provider: Package(pool_package)

    Provider->>Provider: compare package.files with pool_package.files
    alt file lists differ
        Provider->>RepositoryPool: refresh(pool_package)
        RepositoryPool->>RepositoryPool: determine repository_name
        RepositoryPool->>CachedRepository: forget(name, version)
        CachedRepository->>ReleaseCache: forget(canonical_name:version)
        ReleaseCache-->>CachedRepository: removed
        RepositoryPool->>CachedRepository: package(name, version)
        CachedRepository->>ReleaseCache: get_release_info(canonical_name, version)
        ReleaseCache-->>CachedRepository: fresh_release_info
        CachedRepository-->>RepositoryPool: Package(refreshed_package)
        RepositoryPool-->>Provider: Package(refreshed_package)
        Provider->>Provider: add key to _refreshed
    else file lists equal
        Provider->>Provider: keep pool_package
    end

    Provider->>Provider: create DependencyPackage and continue resolution

Class diagram for cache refresh of locked packages

classDiagram
    class Provider {
        +RepositoryPool _pool
        +Callable get_package_from_pool
        +set~tuple~ _refreshed
        +complete_package(package, dependency) DependencyPackage
    }

    class RepositoryPool {
        +list~Repository~ repositories
        +package(name, version, repository_name) Package
        +repository(name) Repository
        +search(query) list~Package~
        +refresh(package) Package
    }

    class CachedRepository {
        +dict _release_cache
        +package(name, version) Package
        +forget(name, version) void
    }

    class Package {
        +str pretty_name
        +Version version
        +list~dict~ files
        +str source_reference
        +str name
    }

    class DependencyPackage {
        +Dependency dependency
        +Package package
    }

    class Dependency {
        +str source_name
    }

    class Repository {
        <<interface>>
        +package(name, version) Package
        +search(query) list~Package~
    }

    Provider --> RepositoryPool : uses
    RepositoryPool --> Repository : aggregates
    Repository <|-- CachedRepository
    Provider --> DependencyPackage : creates
    DependencyPackage --> Package
    DependencyPackage --> Dependency
    RepositoryPool --> Package : refresh(package)
    CachedRepository --> Package : package(name, version)
    CachedRepository --> Package : forget(name, version)

File-Level Changes

Change	Details	Files
Provider now compares locked package files with repository package files and refreshes cached data once per package/source when they differ.	Introduced a per-provider _refreshed set keyed by (name, version, source_name) to track which packages have already triggered a refresh. Adjusted complete_package to optionally bypass the cached pool accessor and use RepositoryPool.package directly when a refresh has already occurred for the package/source tuple. Added logic in complete_package to compare sorted file lists between the locked package and the pool package, calling RepositoryPool.refresh and recording the refresh when they differ, and then building DependencyPackage from the refreshed package.	`src/poetry/puzzle/provider.py` `tests/puzzle/test_provider.py`
RepositoryPool and CachedRepository gained APIs to invalidate cached release info and reload a package from its source repository.	Added RepositoryPool.refresh, which resolves the backing repository from a package's source_reference (defaulting to PyPI), calls forget on CachedRepository instances, and re-fetches the package. Extended CachedRepository with a forget method that evicts a single release entry from its internal _release_cache. Created tests for CachedRepository.get_release_info behavior with and without cache disabled, including cache invalidation via forget.	`src/poetry/repositories/repository_pool.py` `src/poetry/repositories/cached_repository.py` `tests/repositories/test_cached_repository.py`
Existing lock and add command tests were updated to ensure they do not spuriously trigger cache refresh by aligning in-memory package file lists with the lockfile.	Updated lock command tests to capture the package added to the repository and populate its files from the locked repository before executing the command. Updated add command tests to ensure the locked docker package's files are synchronized with the locker's locked_repository when the locked fixture is used.	`tests/console/commands/test_lock.py` `tests/console/commands/test_add.py`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey there - I've reviewed your changes - here's some feedback:

The _refreshed set in Provider grows monotonically for the lifetime of the provider; consider a strategy to bound or periodically clear it (or key it more narrowly) to avoid unbounded memory growth on large or long-running resolves.
In RepositoryPool.refresh, the fallback to the literal "PyPI" when source_reference is missing relies on that exact repository name being present; it may be safer to reuse the same defaulting logic as other pool lookups or make the default repository explicit at construction time to avoid subtle mismatches.
There are two very similar MockCachedRepository test doubles (in tests/puzzle/test_provider.py and tests/repositories/test_cached_repository.py); consider extracting a shared helper or fixture to reduce duplication and keep behavior aligned if one evolves.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `_refreshed` set in `Provider` grows monotonically for the lifetime of the provider; consider a strategy to bound or periodically clear it (or key it more narrowly) to avoid unbounded memory growth on large or long-running resolves.
- In `RepositoryPool.refresh`, the fallback to the literal "PyPI" when `source_reference` is missing relies on that exact repository name being present; it may be safer to reuse the same defaulting logic as other pool lookups or make the default repository explicit at construction time to avoid subtle mismatches.
- There are two very similar `MockCachedRepository` test doubles (in `tests/puzzle/test_provider.py` and `tests/repositories/test_cached_repository.py`); consider extracting a shared helper or fixture to reduce duplication and keep behavior aligned if one evolves.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

… in cache When working on a project with many developers unnecessary diffs in the lockfile due to outdated caches of some developers are an annoying issue. Outdated caches are normally the result of postponed artifact uploads. This change detects such diffs and refreshes the cache automatically.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://github.com/python-poetry/poetry) ([changelog](https://python-poetry.org/history/)) | minor | ` 2.2.1` -> `2.3.0` | --- ### Release Notes <details> <summary>python-poetry/poetry (poetry)</summary> ### [`v2.3.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#230---2026-01-18) [Compare Source](python-poetry/poetry@2.2.1...2.3.0) ##### Added - **Add support for exporting `pylock.toml` files with `poetry-plugin-export`** ([#10677](python-poetry/poetry#10677)). - Add support for specifying build constraints for dependencies ([#10388](python-poetry/poetry#10388)). - Add support for publishing artifacts whose version is determined dynamically by the build-backend ([#10644](python-poetry/poetry#10644)). - Add support for editable project plugins ([#10661](python-poetry/poetry#10661)). - Check `requires-poetry` before any other validation ([#10593](python-poetry/poetry#10593)). - Validate the content of `project.readme` when running `poetry check` ([#10604](python-poetry/poetry#10604)). - Add the option to clear all caches by making the cache name in `poetry cache clear` optional ([#10627](python-poetry/poetry#10627)). - Automatically update the cache for packages where the locked files differ from cached files ([#10657](python-poetry/poetry#10657)). - Suggest to clear the cache if running a command with `--no-cache` solves an issue ([#10585](python-poetry/poetry#10585)). - Propose `poetry init` when trying `poetry new` for an existing directory ([#10563](python-poetry/poetry#10563)). - Add support for `poetry publish --skip-existing` for new Nexus OSS versions ([#10603](python-poetry/poetry#10603)). - Show Poetry's own Python's path in `poetry debug info` ([#10588](python-poetry/poetry#10588)). ##### Changed - **Drop support for Python 3.9** ([#10634](python-poetry/poetry#10634)). - **Change the default of `installer.re-resolve` from `true` to `false`** ([#10622](python-poetry/poetry#10622)). - **PEP 735 dependency groups are considered in the lock file hash** ([#10621](python-poetry/poetry#10621)). - Deprecate `poetry.utils._compat.metadata`, which is sometimes used in plugins, in favor of `importlib.metadata` ([#10634](python-poetry/poetry#10634)). - Improve managing free-threaded Python versions with `poetry python` ([#10606](python-poetry/poetry#10606)). - Prefer JSON API to HTML API in legacy repositories ([#10672](python-poetry/poetry#10672)). - When running `poetry init`, only add the readme field in the `pyproject.toml` if the readme file exists ([#10679](python-poetry/poetry#10679)). - Raise an error if no hash can be determined for any distribution link of a package ([#10673](python-poetry/poetry#10673)). - Require `dulwich>=0.25.0` ([#10674](python-poetry/poetry#10674)). ##### Fixed - Fix an issue where `poetry remove` did not work for PEP 735 dependency groups with `include-group` items ([#10587](python-poetry/poetry#10587)). - Fix an issue where `poetry remove` caused dangling `include-group` references in PEP 735 dependency groups ([#10590](python-poetry/poetry#10590)). - Fix an issue where `poetry add` did not work for PEP 735 dependency groups with `include-group` items ([#10636](python-poetry/poetry#10636)). - Fix an issue where PEP 735 dependency groups were not considered in the lock file hash ([#10621](python-poetry/poetry#10621)). - Fix an issue where wrong markers were locked for a dependency that was required by several groups with different markers ([#10613](python-poetry/poetry#10613)). - Fix an issue where non-deterministic markers were created in a method used by `poetry-plugin-export` ([#10667](python-poetry/poetry#10667)). - Fix an issue where wrong wheels were chosen for installation in free-threaded Python environments if Poetry itself was not installed with free-threaded Python ([#10614](python-poetry/poetry#10614)). - Fix an issue where `poetry publish` used the metadata of the project instead of the metadata of the build artifact ([#10624](python-poetry/poetry#10624)). - Fix an issue where `poetry env use` just used another Python version instead of failing when the requested version was not supported by the project ([#10685](python-poetry/poetry#10685)). - Fix an issue where `poetry env activate` returned the wrong command for `dash` ([#10696](python-poetry/poetry#10696)). - Fix an issue where `data-dir` and `python.installation-dir` could not be set ([#10595](python-poetry/poetry#10595)). - Fix an issue where Python and pip executables were not correctly detected on Windows ([#10645](python-poetry/poetry#10645)). - Fix an issue where invalid template variables in `virtualenvs.prompt` caused an incomprehensible error message ([#10648](python-poetry/poetry#10648)). ##### Docs - Add a warning about `~/.netrc` for Poetry credential configuration ([#10630](python-poetry/poetry#10630)). - Clarify that the local configuration takes precedence over the global configuration ([#10676](python-poetry/poetry#10676)). - Add an explanation in which cases `packages` are automatically detected ([#10680](python-poetry/poetry#10680)). ##### poetry-core ([`2.3.0`](https://github.com/python-poetry/poetry-core/releases/tag/2.3.0)) - Normalize versions ([#893](python-poetry/poetry-core#893)). - Fix an issue where unsatisfiable requirements did not raise an error ([#891](python-poetry/poetry-core#891)). - Fix an issue where the implicit main group did not exist if it was explicitly declared as not having any dependencies ([#892](python-poetry/poetry-core#892)). - Fix an issue where `python_full_version` markers with pre-release versions were parsed incorrectly ([#893](python-poetry/poetry-core#893)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/1654 Co-authored-by: renovate-bot <[email protected]> Co-committed-by: renovate-bot <[email protected]>

github-actions · 2026-01-29T00:43:23Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

sourcery-ai bot reviewed Dec 11, 2025

View reviewed changes

radoering force-pushed the fix-outdated-cache branch from 9a77757 to b12106b Compare December 29, 2025 13:20

radoering merged commit 8a1b011 into python-poetry:main Dec 29, 2025
54 checks passed

radoering mentioned this pull request Jan 17, 2026

release: bump version to 2.3.0 #10695

Merged

Tasssadar mentioned this pull request Jan 20, 2026

Poetry 2.3 breaks caching with private repositories #10698

Closed

github-actions bot locked as resolved and limited conversation to collaborators Jan 29, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

cache: automatic refresh if files of locked package differ from files in cache#10657

cache: automatic refresh if files of locked package differ from files in cache#10657
radoering merged 1 commit intopython-poetry:mainfrom
radoering:fix-outdated-cache

radoering commented Dec 11, 2025 •

edited by sourcery-ai bot

Loading

Uh oh!

sourcery-ai bot commented Dec 11, 2025 •

edited

Loading

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Uh oh!

Uh oh!

github-actions bot commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

Conversation

radoering commented Dec 11, 2025 • edited by sourcery-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Check List

Summary by Sourcery

Uh oh!

sourcery-ai bot commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for automatic cache refresh on package file mismatch

Class diagram for cache refresh of locked packages

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

radoering commented Dec 11, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Dec 11, 2025 •

edited

Loading