Skip to content

TST/DEP: guarding against stealth upgrades in oldestdeps CI #18965

Description

@neutrinoceros

This is intended as a small unit of work to follow up on #18782 (which this requires), but I assume it needs a bit of context.

While solving #18782 (in progress) will obliterate a failure mode in CI (upgrades in transitive dependencies breaking direct ones), the proposed solution (using uv's --resolution=lowest) still implies dynamic dependency resolution before tests runs, and as such, leaves another failure mode open: stealth upgrades.

To simplify, let's assume we maintain package A, which requires packages B and C, and that B itself depends on C. Furthermore

B is available in two versions:

  • 1.0.0 requires C>=0.1.0
  • 2.0.0 requires C>=0.2.0.

Our initial conditions are that A requires B>=1.0.0 and C>=0.1.0. If we resolve the dependency graph to the lowest possible solution, we'll get B==1.0.0 and C==0.1.0, which is what we want to test (our oldest supported dependencies).

Now, if we bump A's requirement on B from >=1.0.0 to >=2.0.0, the new solution to our lowest requirements will contain an upgraded version of C (0.2.0), which is also our de-facto lowest supported version, despite what our metadata declares, so we're now effectively shipping package A with incorrect and misleading metadata. We're also in a situation where we can break backwards compatibility with C==0.1.0 silently, which we'd only learn about when we update our requirement on B again, or, quite possibly, never.
We might also forget to clean associated backward compatibility code that is not reachable anymore. While test coverage measurement could help with the latter, it wouldn't catch the former issue with metadata.

While it will be visible in CI, it would be a hassle to rely soly on maintainer diligence and discipline to catch this class of problems manually every time a PR adds, removes, or updates any dependency (Python included).

The reliable approach is thus to have an automated check for it in CI.
The exact form of such a check is open for discussion, but it could be as simple as comparing the output of pip list against a previous known-good solution.
Note that the correct course of action when a difference is detected would need to be decided on a case-by-case basis. This issue is purely about ensuring that human maintainers are made aware of these upgrades when they happen.

Activity

  1. changed the title [-]TST/DEP: guarding against stealth upgrade in `oldestdeps` CI[/-] [+]TST/DEP: guarding against stealth upgrades in `oldestdeps` CI[/+] on Nov 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions