fix: ops.testing.Container fallback type checking; ty and pyrefly in ci by Vivek1106-04 · Pull Request #2804 · canonical/operator

Vivek1106-04 · 2026-10-07T05:23:01Z

This implements items 1 and 5 from @tonyandrewmeyer's type checker gist, limited to the example charms.

The fix

ops/testing.py imports Container from Scenario and falls back to ops.model.Container when Scenario isn't installed. ty and pyrefly see both imports and treat ops.testing.Container as scenario.Container | ops.model.Container. A Scenario-style call such as testing.Container(name=..., can_connect=True, layers=...) then fails against ops.model.Container.__init__.

The fallback is now guarded with if not TYPE_CHECKING:, so type checkers see only scenario.Container. Runtime behaviour is unchanged: ops.testing.Container is still scenario.Container when Scenario is installed, and ops.model.Container when it isn't.

CI

example-charm-tests.yaml now runs pyrefly and ty after mypy. All three checkers fail the job on errors.

Each checker step syncs the example's own environment with uv sync, installs this branch's Ops and Scenario into it with uv pip install ../.. ../../testing, then runs the checker with uv run --no-sync. The mypy step now works the same way. uv run --with layered the local Ops over the PyPI Ops from the example's lockfile using a .pth file, which ty doesn't follow, so ty would have checked against the PyPI release rather than this branch.
pyrefly and ty get lib as an extra search path when it exists (k8s-3, k8s-4 and k8s-5 fetch charm libs in CI), playing the same role as lib in MYPYPATH.
pyproject.toml has separate mypy, pyrefly (>=1.3) and ty (==0.0.85, pinned because it's pre-1.0) dependency groups, each written to its own requirements file by the workflow.
pyrefly is added to the dev-tooling Dependabot group, alongside ty and mypy.

Testing

Run locally with the same commands as the new workflow steps ("before" is without the ops/testing.py change):

Example	ty before	ty after	pyrefly before	pyrefly after
httpbin-demo	25	0	25	0
k8s-1-minimal	4	0	4	0
k8s-2-configurable	12	0	12	0
k8s-3-postgresql	21	1*	21	1*
k8s-4-action	29	1*	29	1*
k8s-5-observe	32	4*	32	4*
machine-tinyproxy	0	0	0	0

* Only unresolved imports of charm libs, which CI fetches with charmcraft fetch-libs. I didn't fetch them locally. In CI, all three checkers pass on all seven examples.

tox -e lint passes (pyright 0 errors).
tox -e unit -- test/test_testing.py passes.
Checked at runtime that ops.testing.Container is scenario.Container with Scenario installed, and is ops.model.Container with Scenario blocked.

Fixes #2803.

ops.testing imports Container from Scenario, falling back to ops.model.Container when Scenario is not installed. ty and pyrefly saw both imports and treated ops.testing.Container as a union, so every Scenario-style testing.Container(...) call in a charm's tests failed to type check. Guard the fallback with TYPE_CHECKING. Runtime behaviour is unchanged. Also type check the example charms with pyrefly and ty in CI, next to mypy. The ty step does not fail the job, as ty is pre-1.0. Fixes canonical#2803

james-garner-canonical

Thanks, a few questions and suggestions.

james-garner-canonical · 2026-10-08T00:39:15Z

+    # Hidden from type checkers so that they see only scenario.Container,
+    # rather than a union of scenario.Container and ops.model.Container.
+    if not TYPE_CHECKING:
+        from .model import Container


This makes sense with how we want this module to be used. I tried to think of a way this could be breaking for someone -- it's a type checking change only, so I guess they'd have to be using testing.Container somewhere that you expect it to be model.Container, expecting the union of testing.Container | model.Container, and the narrowing it to just model.Container, which seems vanishingly unlikely.

Use one dependency group and requirements file per type checker, make ty a blocking step, and drop search paths that pyrefly doesn't need. ty still needs the local Ops and Scenario paths because it doesn't follow the .pth file that uv run uses for --with packages.

Vivek1106-04 · 2026-10-08T03:48:33Z

@tonyandrewmeyer thanks! Split it into separate mypy, pyrefly and ty groups, each with its own requirements file. ty is a normal blocking step now and passes on all the examples. I kept it pinned since it's still 0.0.x.

Vivek1106-04 · 2026-10-08T03:48:35Z

@james-garner-canonical thanks for the review!

Search paths: pyrefly didn't need them, so they're gone. ty does, because uv run --with adds packages through a .pth file ty doesn't follow, so it ends up checking against the PyPI ops in the charm's venv. Added a comment in the workflow.
lib: k8s-3, k8s-4 and k8s-5 fetch charm libs in CI, so it exists there. Both tools fail on a missing path, hence the check.

james-garner-canonical · 2026-10-08T04:07:49Z

Search paths: pyrefly didn't need them, so they're gone. ty does, because uv run --with adds packages through a .pth file ty doesn't follow, so it ends up checking against the PyPI ops in the charm's venv. Added a comment in the workflow.

Interesting, I don't think we want the PyPI version of Ops to end up in the charm's venv. Shouldn't uv run --with resolve out such that we only get the local version in the venv?

lib: k8s-3, k8s-4 and k8s-5 fetch charm libs in CI, so it exists there. Both tools fail on a missing path, hence the check.

Ah makes sense, I missed that we fetch these in CI.

Vivek1106-04 · 2026-10-08T04:13:46Z

@james-garner-canonical the charm's own lockfile pins ops 3.7.0 from PyPI, so uv run syncs that into the venv first. --with doesn't replace it, it layers local Ops on top via a .pth file. Fine at runtime and for mypy/pyrefly, but ty doesn't follow it. Same as the existing mypy step.

If you'd rather avoid PyPI Ops entirely, I can switch to uv sync + uv pip install ../.. ../../testing, which lets ty work without the search paths (tested locally). Happy to do that if you prefer.

james-garner-canonical · 2026-10-08T05:52:25Z

@james-garner-canonical the charm's own lockfile pins ops 3.7.0 from PyPI, so uv run syncs that into the venv first. --with doesn't replace it, it layers local Ops on top via a .pth file. Fine at runtime and for mypy/pyrefly, but ty doesn't follow it. Same as the existing mypy step.

Thanks for explaining, I hadn't realised that that's how uv run --with was implemented.

If you'd rather avoid PyPI Ops entirely, I can switch to uv sync + uv pip install ../.. ../../testing, which lets ty work without the search paths (tested locally). Happy to do that if you prefer.

I tested this locally too and I agree that this is the right approach. If you can switch them all to use uv sync, uv pip install ../../ ../../testing -r ${{ ... }}-dependencies.txt, and uv run --no-sync that would be great, thanks!

Sync each example charm's venv, then replace its PyPI Ops with the local ops and ops-scenario packages via uv pip install, and run the checkers with uv run --no-sync. ty no longer needs extra search paths to see the local source.

Vivek1106-04 · 2026-10-08T06:42:18Z

@james-garner-canonical done in 5f7a2f0. mypy, pyrefly and ty now each run uv sync, then uv pip install ../.. ../../testing -r ../../<checker>-requirements.txt, then uv run --no-sync. ty doesn't need the extra search paths for Ops/Scenario anymore, only lib when it exists. Ran it locally on all seven examples: the venv has Ops 3.9.0.dev0 and all three checkers pass.

james-garner-canonical

Thanks @Vivek1106-04, this looks good to me.

The new pyrefly dependency group would otherwise fall through to the runtime group (or its own PR for a major version), unlike ty and mypy. Co-Authored-By: Claude Opus 5.5 <[email protected]>

tonyandrewmeyer

Thanks for this. It's unfortunate that we need to do this, but it seems like this will improve things for some cases without making it worse for anyone else. We can fix it properly in 4.0.

Vivek1106-04 mentioned this pull request Oct 7, 2026

Type check example charms with newer type checkers #2803

Closed

Vivek1106-04 marked this pull request as ready for review October 7, 2026 05:26

tonyandrewmeyer requested review from james-garner-canonical and tonyandrewmeyer October 8, 2026 00:19

tonyandrewmeyer reviewed Oct 8, 2026

View reviewed changes

Comment thread pyproject.toml Resolved

Comment thread .github/workflows/example-charm-tests.yaml Resolved

Comment thread .github/workflows/example-charm-tests.yaml Resolved

james-garner-canonical reviewed Oct 8, 2026

View reviewed changes

Vivek1106-04 requested review from james-garner-canonical and tonyandrewmeyer October 8, 2026 03:49

james-garner-canonical approved these changes Oct 8, 2026

View reviewed changes

ci: group pyrefly with the other type checkers in Dependabot

3c73523

The new pyrefly dependency group would otherwise fall through to the runtime group (or its own PR for a major version), unlike ty and mypy. Co-Authored-By: Claude Opus 5.5 <[email protected]>

tonyandrewmeyer approved these changes Oct 11, 2026

View reviewed changes

james-garner-canonical changed the title ~~fix: hide the ops.testing.Container fallback from type checkers~~ fix: ops.testing.Container fallback type checking; ty and pyrefly in ci Oct 12, 2026

james-garner-canonical merged commit ed42537 into canonical:main Oct 12, 2026
61 checks passed

Conversation

Vivek1106-04 commented Oct 7, 2026 • edited by james-garner-canonical Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The fix

CI

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

james-garner-canonical left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

james-garner-canonical Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

Vivek1106-04 commented Oct 8, 2026

Uh oh!

Vivek1106-04 commented Oct 8, 2026

Uh oh!

james-garner-canonical commented Oct 8, 2026

Uh oh!

Vivek1106-04 commented Oct 8, 2026

Uh oh!

james-garner-canonical commented Oct 8, 2026

Uh oh!

Vivek1106-04 commented Oct 8, 2026

Uh oh!

james-garner-canonical left a comment

Choose a reason for hiding this comment

Uh oh!

tonyandrewmeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Vivek1106-04 commented Oct 7, 2026 •

edited by james-garner-canonical

Loading