Repository navigation
validate-kits.py fails on a gitignored __pycache__ dir under acq-kits/ #383
Description
Defect
make validate-kits fails with:
FAIL:
- __pycache__: missing spec.yaml
whenever a __pycache__/ directory exists under integrations/isolation/acq-kits/ (created any time a Python tool imports a module from that tree). __pycache__/ is gitignored (.gitignore:2), so it never appears in git status and CI (fresh checkout) never sees it — but it breaks every local make validate-kits run after any Python import touches that path, and the failure is confusing (looks like a kit is malformed).
Root cause
integrations/isolation/acq-kits/validate-kits.py:319:
kit_dirs = sorted(d for d in kits_dir.iterdir() if d.is_dir()) if kits_dir.exists() else []It treats every subdirectory as a kit and then requires a spec.yaml. __pycache__ (and any other non-kit dir) is not a kit.
Fix
Skip non-kit dirs in the iterdir() filter — at minimum names starting with . or __ (or, more principled, only descend into dirs listed in kits.yaml, or dirs that contain a spec.yaml). E.g.:
kit_dirs = sorted(
d for d in kits_dir.iterdir()
if d.is_dir() and not d.name.startswith((".", "__"))
)Add a regression test that a __pycache__ (or dotdir) under the kits dir does not fail validation.
Impact
Local-only DX papercut (CI is unaffected because a fresh checkout has no __pycache__), but it repeatedly forces a manual "move the pycache out of the way" workaround during local kit work. Low risk, small fix.
AI-assisted (OpenCode). Found while running make validate-kits during the prime-agent/catalog work.
Activity
wz-gsa commentedon Sep 29, 2026
Hit this again five weeks later, independently, while preparing a combined ruff-pin PR — the commit was blocked by the
validate-acq-kitshook and I spent a few minutes assuming my own change had broken something. Adding the parts the original report didn't have, since it's still open and this is the second occurrence.Which test creates it, and a clean reproduction
The original says "any time a Python tool imports a module from that tree." It's narrower and more predictable than that:
scripts/tests/test_validate_kits_*.pyimport the validator by path, so CPython writes the bytecode cache next to the source —From a clean clone of
mainat5da8188:So it isn't incidental — running the repo's own test suite reliably breaks the repo's own pre-commit hook. Anyone who runs
pytestbeforegit commithits it, which is the normal order.Why CI stays green, confirmed structurally
The original says CI is unaffected because a fresh checkout has no
__pycache__. That's right, and it's stronger than luck: the two steps are in different jobs with separate checkouts —pytestinpython(ci.yml:21) andmake validate-kitsinvalidate(ci.yml:151). So no ordering change within a job could expose it either.That's worth stating because it means CI can never regress on this, and the fix is purely a local-DX fix. It also means a contributor's first encounter is a confusing pre-commit failure naming a kit that doesn't exist.
The proposed fix still looks right
The one-line
startswith((".", "__"))filter at:319is what I'd independently arrived at. Of the alternatives in the original:PYTHONDONTWRITEBYTECODE=1in the test invocation fixes the symptom but leaves the walk fragile to any other stray directory, and "only descend into dirs listed inkits.yaml" would change the validator's relationship to the registry — right now the walk is the source of truth andkits.yamlis cross-checked against it, so inverting that is a design change rather than a bug fix.One suggestion on the regression test: assert the exit code, not just the absence of the message. A test that only greps stdout would pass if the validator silently skipped every kit.
One adjacent observation, not part of this issue
validate-kits.py:320-322:That returns success on zero evidence — the inverse of this bug, and the shape this repo has been bitten by before. Not proposing anything here; flagging it because whoever touches
:319will be one line away from it.AI-assisted (OpenCode), advisory. Reproduction is from a clean clone of
mainat5da8188; the fix suggestion is untested.feat(patterns): add paseo kit identity gate for a stale host forwarder (