Skip to content

Commit 13b760a

Browse files
CrazyBaranclaude
andauthored
Fix #4345 (3/4): CI guard requiring version bumps on bundled extension changes (#4395)
* ci(extensions): guard bundled extension changes behind a version bump Bundled extensions only reach existing installs through a version bump: `specify extension update` compares the semver in extensions/catalog.json against the installed copy and reports "Up to date" whenever they match. Content changes shipped without a bump go silently stale on every project that already installed the extension (#4345). This guard turns "please remember to bump" into a merge requirement. - `.github/scripts/check_extension_version_bump.py` fails a PR that changes files under `extensions/<id>/` for a catalog-listed extension without increasing that extension's `extension.yml` version (PEP 440 comparison, the same semantics `extension update` uses), and requires `extensions/catalog.json` to stay in sync with each manifest. Non-catalog extensions (the `selftest` fixture and the `template` scaffold) are exempt. - `extension-version-guard.yml` runs the check on pull requests touching `extensions/**`, diffing the PR base against HEAD. - Contract tests pin the working-tree half of the invariant (catalog/manifest version sync, bundled entries shipping a directory); guard-script tests pin the failure behavior against real throwaway git repositories so a parsing change cannot silently disable the guard while CI stays green. Split out of #4351 (part 3 of the series requested in review); refs #4345. Assisted-by: Claude Code (model: claude-fable-5) Co-Authored-By: Claude Fable 5 <[email protected]> * fix(ci): make the extension version guard usable as a required check and quoting-proof Address Copilot review round 1 on #4395. Workflow: drop the `paths: extensions/**` filter. A required status check that is skipped by path filtering stays in "Expected" state and blocks every PR that does not touch extensions/**, which defeats the point of making the guard a merge requirement. The job now runs on every pull request; the script already reports success when nothing under extensions/ changed, so unrelated PRs pass in one short job. Script: read the changed-path list with `git diff --name-only -z`. With git's default core.quotePath, a path containing non-ASCII or control characters is C-quoted with the quotes included (`"extensions/demo/caf\303\251.txt"`), so its first component was no longer `extensions` and an unbumped change to such a file escaped the guard. NUL-delimited output is emitted verbatim; paths are decoded with surrogateescape so an undecodable byte cannot crash the check, and only the ASCII `extensions/<id>/` prefix is ever interpreted. Tests: pin both behaviors. The non-ASCII case fails against the previous script and passes now; the no-extension-changes case backs the workflow change. core.quotePath is pinned to true in the fixture repo so the regression exercises the quoting path even where a developer's global config disables it. Refs #4345 Assisted-by: Claude Code (model: claude-fable-5-1) Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(ci): validate a new extension's version even without a base to compare Address @mnriem's review on #4395. For a brand-new extension the guard returned early at "no base manifest" before ever parsing the head version, and the catalog check only tests string equality. A new extension shipped with e.g. `not-a-version` in both extension.yml and catalog.json therefore passed the guard, although ExtensionManifest rejects a version packaging cannot parse (ValidationError "Invalid version") and `extension update` skips catalog entries whose version is invalid - the extension would be uninstallable and never updatable. Parse the head version before the new-extension early return and fail closed with a dedicated message when it is not PEP 440. The base version is now parsed separately, so an unparseable base still fails with the "could not compare versions" message. Tests: add test_new_extension_with_unparseable_version_fails (fails against the previous script, passes now) and point the existing unparseable-version test at the new head-version message, which fires first for that scenario. Refs #4345 Assisted-by: Claude Code (model: claude-fable-5-1) Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(ci): diff the extension version guard against the merge commit's first parent The guard diffed github.event.pull_request.base.sha against the checked-out pull-request merge commit. Those two are not built from the same base: the payload SHA is the base-branch tip from when the PR was opened and is not refreshed when the base moves, while refs/pull/N/merge is rebuilt against the current tip. Any base-branch drift between the two is then attributed to the PR. Observed on #4395 itself: base.sha was main as of Sep 1 (3eec154), the merge commit was built on main as of Sep 3 (db64869), and two unbumped bundled-extension changes merged in between (#4413 touching agent-context, #4286 touching git) failed the guard on a PR that never touched either. Fetch the merge commit with depth 2 so both parents are available and diff against HEAD^1, which is by construction the base the merge was built on. Fail loudly if HEAD is not a merge commit. The script's CLI is unchanged; a regression test reproduces the trap in a throwaway repo (the stale base blames the drift on the PR, HEAD^1 does not). Refs #4345 Assisted-by: Claude Code (model: claude-fable-5-1) Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(ci): validate every catalog version, not only those of changed extensions Address the "needs a closer look" items from the Copilot reviews on #4395. Invariant 2 skipped catalog entries without an in-repo directory before parsing their version, and never parsed the in-repo manifest version at all - PEP 440 validation lived only in Invariant 1, which sees just the IDs with changed files under extensions/<id>/. Two gaps followed: - a hosted (catalog-only) entry could carry `"version": "not-a-version"` and pass, although `extension update` skips exactly such entries when packaging.Version fails; - promoting an existing uncataloged directory by adding only its catalog entry never entered Invariant 1, so a matching invalid string in manifest and catalog passed the plain equality check. Invariant 2 now runs over every catalog entry: the catalog version must be a non-empty string that parses as PEP 440, and for entries with an in-repo directory the manifest version must parse as well before the equality check. The catalog-side parse alone closes the promotion gap (a valid catalog string plus equality forces a valid manifest string); the manifest parse makes the failure name the manifest precisely. Tests: `_run_guard` now omits the head argument by default, matching the workflow's one-argument invocation, with a separate test for the optional HEAD_REF; new cases cover the hosted-entry and both promotion shapes (the two invalid ones fail against the previous script) plus a valid promotion that must keep passing. The working-tree contract test gains the matching every-catalog-version-parses check. Refs #4345 Assisted-by: Claude Code (model: claude-fable-5-1) Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Jakub Baranowski <[email protected]> Co-authored-by: Claude Fable 5 <[email protected]>
1 parent b600576 commit 13b760a

5 files changed

Lines changed: 731 additions & 0 deletions

File tree

Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
#!/usr/bin/env python3
2+
"""Fail a PR that changes bundled extension content without a version bump.
3+
4+
Update offers from `specify extension update` are version-driven: an
5+
extension is offered (and installed) only when the semver in
6+
`extensions/catalog.json` exceeds the installed copy's registered
7+
version. A content change shipped without a version bump is therefore
8+
never delivered automatically (#4345) — a bump is what makes a change
9+
actually reach existing installs, and this guard is what makes the bump
10+
non-optional.
11+
12+
This check enforces two invariants on the extensions listed in
13+
`extensions/catalog.json`:
14+
15+
1. Any change to a file under `extensions/<id>/` must increase the
16+
`version:` in that extension's `extension.yml` (PEP 440 comparison,
17+
the same semantics `extension update` uses), and the resulting
18+
version must itself parse as PEP 440 — including for a brand-new
19+
extension, since the CLI rejects a manifest whose version it cannot
20+
parse.
21+
2. Every `version` in `extensions/catalog.json` must parse as PEP 440
22+
(`extension update` skips entries it cannot parse), and for entries
23+
with an in-repo directory it must equal a likewise-valid
24+
`extension.version` in the manifest (the catalog is what update
25+
checks compare against, and the update preflight rejects a manifest
26+
whose version differs from the catalog's). This runs over every
27+
catalog entry, so catalog-only (hosted) entries and catalog-only
28+
promotions of existing directories are covered too.
29+
30+
Usage:
31+
check_extension_version_bump.py BASE_REF [HEAD_REF]
32+
33+
BASE_REF is a git ref/SHA for the PR base (must be fetchable with
34+
`git show`). HEAD_REF defaults to the working tree's HEAD. Exits 0 when
35+
all invariants hold, 1 otherwise, printing one line per violation.
36+
37+
Extensions under `extensions/` that are not in the catalog (the
38+
`selftest` fixture and the `template` scaffold) are exempt: no update
39+
flow is driven by their versions.
40+
"""
41+
42+
from __future__ import annotations
43+
44+
import json
45+
import subprocess
46+
import sys
47+
from pathlib import PurePosixPath
48+
49+
import yaml
50+
from packaging.version import InvalidVersion, Version
51+
52+
EXTENSIONS_ROOT = "extensions"
53+
CATALOG_PATH = f"{EXTENSIONS_ROOT}/catalog.json"
54+
55+
56+
def _changed_paths(base_ref: str, head_ref: str) -> list[str]:
57+
"""Paths under extensions/ that differ between *base_ref* and *head_ref*.
58+
59+
Uses NUL-delimited output (``-z``): without it git C-quotes any path
60+
containing non-ASCII or control characters (``"extensions/x/caf\\303\\251"``,
61+
quotes included), so the leading component would no longer equal
62+
``extensions`` and that change would silently escape the guard. Paths
63+
are decoded with surrogateescape so an undecodable byte can never crash
64+
the check; only the ASCII ``extensions/<id>/`` prefix is interpreted.
65+
"""
66+
raw = subprocess.run(
67+
[
68+
"git", "diff", "--name-only", "-z", "--no-renames",
69+
base_ref, head_ref, "--", EXTENSIONS_ROOT,
70+
],
71+
check=True,
72+
capture_output=True,
73+
).stdout
74+
return [
75+
chunk.decode("utf-8", errors="surrogateescape")
76+
for chunk in raw.split(b"\0")
77+
if chunk
78+
]
79+
80+
81+
def _show(ref: str, path: str) -> str | None:
82+
"""Return the file's content at *ref*, or None when absent there."""
83+
result = subprocess.run(
84+
["git", "show", f"{ref}:{path}"], capture_output=True, text=True
85+
)
86+
return result.stdout if result.returncode == 0 else None
87+
88+
89+
def _manifest_version(manifest_text: str, origin: str) -> str:
90+
data = yaml.safe_load(manifest_text)
91+
if not isinstance(data, dict) or not isinstance(data.get("extension"), dict):
92+
raise ValueError(f"{origin}: manifest is not a mapping with an 'extension' block")
93+
version = data["extension"].get("version")
94+
if not isinstance(version, str) or not version.strip():
95+
raise ValueError(f"{origin}: extension.version is missing or not a string")
96+
return version.strip()
97+
98+
99+
def main(argv: list[str]) -> int:
100+
if len(argv) < 2 or len(argv) > 3:
101+
print(__doc__, file=sys.stderr)
102+
return 2
103+
base_ref = argv[1]
104+
head_ref = argv[2] if len(argv) == 3 else "HEAD"
105+
106+
catalog_text = _show(head_ref, CATALOG_PATH)
107+
if catalog_text is None:
108+
print(f"::error::{CATALOG_PATH} is missing at {head_ref}")
109+
return 1
110+
catalog = json.loads(catalog_text)
111+
catalog_entries = catalog.get("extensions", {})
112+
113+
errors: list[str] = []
114+
115+
# -- Invariant 1: content change requires a version bump ---------------
116+
changed_ids = {
117+
parts[1]
118+
for path in _changed_paths(base_ref, head_ref)
119+
if len(parts := PurePosixPath(path).parts) >= 3 and parts[0] == EXTENSIONS_ROOT
120+
}
121+
122+
for ext_id in sorted(changed_ids):
123+
if ext_id not in catalog_entries:
124+
continue # not driven by `extension update` (selftest, template)
125+
manifest_path = f"{EXTENSIONS_ROOT}/{ext_id}/extension.yml"
126+
head_manifest = _show(head_ref, manifest_path)
127+
if head_manifest is None:
128+
continue # extension removed in this PR
129+
try:
130+
head_version = _manifest_version(head_manifest, f"{head_ref}:{manifest_path}")
131+
except ValueError as exc:
132+
errors.append(str(exc))
133+
continue
134+
135+
# Parse the head version before the new-extension early return: the
136+
# CLI's ExtensionManifest rejects a version packaging cannot parse and
137+
# `extension update` skips catalog entries whose version is invalid,
138+
# so a new extension shipped with e.g. "not-a-version" in both places
139+
# would be uninstallable even though the catalog check below (plain
140+
# string equality) passes. Same PEP 440 semantics as the CLI, so
141+
# prereleases and other accepted forms are handled identically.
142+
try:
143+
head_parsed = Version(head_version)
144+
except InvalidVersion as exc:
145+
errors.append(
146+
f"{manifest_path}: extension.version {head_version!r} is not a valid "
147+
f"PEP 440 version ({exc}); the CLI rejects this manifest."
148+
)
149+
continue
150+
151+
base_manifest = _show(base_ref, manifest_path)
152+
if base_manifest is None:
153+
continue # new extension; any valid initial version is fine
154+
try:
155+
base_version = _manifest_version(base_manifest, f"{base_ref}:{manifest_path}")
156+
except ValueError as exc:
157+
errors.append(str(exc))
158+
continue
159+
160+
# Compare with the same PEP 440 semantics the extension update and
161+
# install code use, so prereleases and other accepted forms cannot
162+
# bypass the guard (e.g. 2.0.0 -> 1.0.0rc1 is a downgrade). An
163+
# unparseable base version fails closed.
164+
try:
165+
base_parsed = Version(base_version)
166+
except InvalidVersion as exc:
167+
errors.append(
168+
f"{manifest_path}: could not compare versions "
169+
f"{base_version!r} -> {head_version!r}: {exc}"
170+
)
171+
continue
172+
if head_parsed <= base_parsed:
173+
errors.append(
174+
f"{manifest_path}: files under {EXTENSIONS_ROOT}/{ext_id}/ changed but "
175+
f"extension.version did not increase ({base_version} -> {head_version}). "
176+
f"Installed copies only receive changes when the version is bumped."
177+
)
178+
179+
# -- Invariant 2: catalog versions are valid and match the manifests ----
180+
# Runs over every catalog entry, changed or not: a catalog-only entry
181+
# (hosted elsewhere) never has files under extensions/<id>/, and a
182+
# catalog-only promotion of an existing uncataloged directory never
183+
# enters Invariant 1, so neither would otherwise have its version parsed.
184+
for ext_id, entry in sorted(catalog_entries.items()):
185+
catalog_version = entry.get("version") if isinstance(entry, dict) else None
186+
if not isinstance(catalog_version, str) or not catalog_version.strip():
187+
errors.append(f"{CATALOG_PATH}: entry '{ext_id}' has no string 'version'")
188+
continue
189+
# `extension update` skips a catalog entry whose version packaging
190+
# cannot parse, so an invalid catalog version is never offered.
191+
try:
192+
Version(catalog_version)
193+
except InvalidVersion as exc:
194+
errors.append(
195+
f"{CATALOG_PATH}: entry '{ext_id}' version {catalog_version!r} is not a "
196+
f"valid PEP 440 version ({exc}); `extension update` skips such entries."
197+
)
198+
continue
199+
200+
manifest_path = f"{EXTENSIONS_ROOT}/{ext_id}/extension.yml"
201+
head_manifest = _show(head_ref, manifest_path)
202+
if head_manifest is None:
203+
continue # catalog-only entry (e.g. hosted elsewhere); version checked above
204+
try:
205+
manifest_version = _manifest_version(head_manifest, f"{head_ref}:{manifest_path}")
206+
except ValueError as exc:
207+
errors.append(str(exc))
208+
continue
209+
try:
210+
Version(manifest_version)
211+
except InvalidVersion as exc:
212+
errors.append(
213+
f"{manifest_path}: extension.version {manifest_version!r} is not a valid "
214+
f"PEP 440 version ({exc}); the CLI rejects this manifest."
215+
)
216+
continue
217+
if catalog_version != manifest_version:
218+
errors.append(
219+
f"{CATALOG_PATH}: entry '{ext_id}' has version {catalog_version!r} but "
220+
f"{manifest_path} declares {manifest_version!r}. `extension update` "
221+
f"compares against the catalog, so the two must move together."
222+
)
223+
224+
for error in errors:
225+
print(f"::error::{error}")
226+
if not errors:
227+
print("Extension version guard: all invariants hold.")
228+
return 1 if errors else 0
229+
230+
231+
if __name__ == "__main__":
232+
sys.exit(main(sys.argv))
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Extension Version Guard
2+
3+
permissions:
4+
contents: read
5+
6+
# Bundled extensions only reach existing installs through a version bump:
7+
# `specify extension update` compares the semver in extensions/catalog.json
8+
# against the installed copy and reports "Up to date" whenever they match.
9+
# Content changes shipped without a bump go silently stale on every
10+
# project that already installed the extension (#4345). This guard turns
11+
# "please remember to bump" into a merge requirement.
12+
#
13+
# Deliberately no `paths:` filter: a required status check that is skipped
14+
# by path filtering stays in "Expected" state and blocks every PR that does
15+
# not touch extensions/**. The check runs on every pull request instead and
16+
# the script reports success when nothing under extensions/ changed.
17+
on:
18+
pull_request:
19+
20+
jobs:
21+
version-bump:
22+
runs-on: ubuntu-latest
23+
steps:
24+
- name: Checkout
25+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
26+
with:
27+
# The pull_request checkout is GitHub's merge of the PR head into
28+
# the base tip. Depth 2 also fetches both parents of that merge
29+
# commit, so HEAD^1 - the base the merge was actually built on -
30+
# is available to diff against.
31+
fetch-depth: 2
32+
33+
- name: Set up Python
34+
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
35+
with:
36+
python-version: "3.14"
37+
38+
- name: Install check dependencies
39+
run: python -m pip install --quiet pyyaml packaging
40+
41+
# Diff against the merge commit's first parent, NOT against
42+
# github.event.pull_request.base.sha. The payload SHA is the base tip
43+
# from when the PR was opened and is not refreshed when the base branch
44+
# moves, while refs/pull/N/merge is rebuilt against the current tip.
45+
# Diffing the stale SHA against the fresh merge commit attributes
46+
# unrelated base-branch drift to the PR (observed on #4395: two
47+
# unbumped extension changes merged to main were blamed on a PR that
48+
# never touched them). HEAD^1 is by construction the base the merge
49+
# was built on.
50+
- name: Check bundled extension version bumps
51+
run: |
52+
set -euo pipefail
53+
if ! git rev-parse --verify --quiet HEAD^2 >/dev/null; then
54+
echo "::error::HEAD is not the pull request merge commit; cannot determine the PR base"
55+
exit 1
56+
fi
57+
python .github/scripts/check_extension_version_bump.py HEAD^1

‎extensions/EXTENSION-DEVELOPMENT-GUIDE.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -625,6 +625,13 @@ See the [Extension Publishing Guide](EXTENSION-PUBLISHING-GUIDE.md) for detailed
625625
- **MAJOR**: Breaking changes
626626
- **MINOR**: New features
627627
- **PATCH**: Bug fixes
628+
- **Bump on every content change**: update offers from `specify extension
629+
update` are version-driven, so a content change shipped without a
630+
version bump is never delivered automatically to already-installed
631+
copies. For the bundled extensions in this repository the bump is
632+
enforced by CI (`extension-version-guard.yml`): a PR that changes
633+
files under `extensions/<id>/` must also bump that extension's
634+
`extension.yml` version and keep `extensions/catalog.json` in sync.
628635

629636
### Security
630637

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
"""Contract tests: bundled extension versions must stay in sync with the catalog.
2+
3+
``specify extension update`` decides whether an installed extension needs
4+
updating by comparing the semver in ``extensions/catalog.json`` against the
5+
installed copy's registered version, and its preflight rejects a manifest
6+
whose version differs from the catalog's. A catalog entry that drifts from
7+
its ``extension.yml`` therefore either hides updates from every installed
8+
copy or makes every offered update fail validation (#4345).
9+
10+
The companion "content change requires a version bump" rule needs the git
11+
diff of a PR and lives in CI
12+
(``.github/scripts/check_extension_version_bump.py`` via the
13+
``extension-version-guard.yml`` workflow); this test enforces the half that
14+
is checkable from a plain working tree.
15+
"""
16+
17+
from __future__ import annotations
18+
19+
import json
20+
from pathlib import Path
21+
22+
import pytest
23+
import yaml
24+
from packaging.version import InvalidVersion, Version
25+
26+
REPO_ROOT = Path(__file__).parents[2]
27+
EXTENSIONS_ROOT = REPO_ROOT / "extensions"
28+
29+
30+
def _catalog_entries() -> dict[str, dict]:
31+
catalog = json.loads((EXTENSIONS_ROOT / "catalog.json").read_text(encoding="utf-8"))
32+
return catalog["extensions"]
33+
34+
35+
def _manifest_version(ext_id: str) -> str:
36+
manifest_path = EXTENSIONS_ROOT / ext_id / "extension.yml"
37+
data = yaml.safe_load(manifest_path.read_text(encoding="utf-8"))
38+
return data["extension"]["version"]
39+
40+
41+
def test_catalog_lists_extensions():
42+
assert _catalog_entries(), "expected at least one extension in extensions/catalog.json"
43+
44+
45+
@pytest.mark.parametrize("ext_id", sorted(_catalog_entries()))
46+
def test_catalog_version_matches_manifest(ext_id: str):
47+
entry = _catalog_entries()[ext_id]
48+
manifest_path = EXTENSIONS_ROOT / ext_id / "extension.yml"
49+
if not manifest_path.is_file():
50+
pytest.skip(f"'{ext_id}' has no in-repo extension directory")
51+
assert entry.get("version") == _manifest_version(ext_id), (
52+
f"extensions/catalog.json entry '{ext_id}' and {manifest_path.relative_to(REPO_ROOT)} "
53+
f"declare different versions - `specify extension update` compares against the "
54+
f"catalog, so the two must move together"
55+
)
56+
57+
58+
@pytest.mark.parametrize("ext_id", sorted(_catalog_entries()))
59+
def test_bundled_entries_ship_an_extension_directory(ext_id: str):
60+
entry = _catalog_entries()[ext_id]
61+
if not entry.get("bundled"):
62+
pytest.skip(f"'{ext_id}' is not marked bundled")
63+
assert (EXTENSIONS_ROOT / ext_id / "extension.yml").is_file(), (
64+
f"catalog marks '{ext_id}' as bundled but extensions/{ext_id}/extension.yml is missing"
65+
)
66+
67+
68+
@pytest.mark.parametrize("ext_id", sorted(_catalog_entries()))
69+
def test_catalog_version_is_valid_pep440(ext_id: str):
70+
"""`extension update` parses each catalog version with packaging and skips
71+
entries it cannot parse, so every entry - bundled or hosted - must carry
72+
a valid version, independent of whether it has an in-repo directory."""
73+
version = _catalog_entries()[ext_id].get("version")
74+
assert isinstance(version, str) and version.strip(), (
75+
f"extensions/catalog.json entry '{ext_id}' has no string version"
76+
)
77+
try:
78+
Version(version)
79+
except InvalidVersion as exc:
80+
pytest.fail(
81+
f"extensions/catalog.json entry '{ext_id}' version {version!r} is not a valid "
82+
f"PEP 440 version ({exc}); `extension update` would skip it"
83+
)

0 commit comments

Comments
 (0)