Skip to content

Commit f8ec601

Browse files
luisleo526ci-cxclaude
authored
ci(release): the permission-claim text guard also catches backslash-wrapped, bytes-decoded and fullwidth respellings (PROBE-FU) [replaces #493] (#506)
* ci(release): the permission-claim text guard also catches backslash-wrapped, bytes-decoded and fullwidth respellings (PROBE-FU, sol P2 1/2/4) The guard in WorkflowPinTests.assert_ruleset_visibility_documentation fails when the gate source, docs/ci.md or release.yml state that the App needs "administration: read". A read of the probe found three respellings it accepted: a Markdown hard break written as a backslash before the newline, a bytes constant (b"administr\x61tion: re\x61d", with or without .decode()), and fullwidth compatibility letters. The collapsed view now treats backslash-newline like whitespace, bytes constants are decoded into the scanned strings, and every view is NFKC-normalised before case folding. Three new tests (12 rows) each read the real artifacts, assert the unmodified baseline passes, mutate a copy and require the specific permission-claim AssertionError; removing only the matching handling turns its rows red. Not closed, by design: YAML double-quoted scalars with escapes in release.yml (sol P2 3) and runtime string construction (sol P3); a static text guard cannot decide them, and the exact-output pins on the refusal text remain the decisive protection. No existing assertion, control or forbidden pattern is changed. The rewritten `view` tuple is code, not a documentation or comment sentence. Lane: ci-fu-pfuguard (Codex gpt-6.1-sol). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * ci(release): the permission-claim guard scans the casefold-only views as well as the NFKC views (PROBE-FU round 2, sol R237 P2) The first PROBE-FU commit normalised every scanned view with NFKC and replaced the casefold-only views. For an input such as `Needs administration: read` followed by the trade mark sign, the old guard matched (the sign is a non-word character after `read`), but after NFKC the text is `readtm` and the word boundary no longer matches, so the claim was accepted: the change made the guard weaker for that input. Each text is now checked in four views (the two original views casefolded, and both NFKC+casefolded), so normalisation can only add matches. A restoration control appends the trade mark case to the gate source, docs/ci.md and the workflow text (direct and line-wrapped, six rows), and a removal mutant drops the casefold-only scan and must break all six rows. No existing assertion or control changes; no documentation or comment sentence is deleted or rewritten. Lane: ci-fu-pfuguard2 (Codex gpt-6.1-sol). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * ci: release ruleset probe lists every branch ruleset (measurement only, gate unchanged) [replaces #489] (#505) * ci(release): the ruleset probe lists every branch ruleset as visible or hidden, even when the gate verdict is a refusal (CR-245 F1 measurement) The one dispatch of release-ruleset-probe.yml from main (run 40083495967) refused on the first branch ruleset of the inventory, the organization ruleset `default` (13094758): "returned missing or invalid bypass_actors (token cannot see this ruleset's bypass_actors)". The verdict reader stops at the first failure, so the run did not say whether the release App token can see bypass_actors on the repository ruleset that is the real barrier (21476386), nor on any other. The probe now first prints one line per branch ruleset of the includes_parents inventory (id, name, source_type, source, enforcement, updated_at, bypass_actors visible with the actor count or hidden, applies to main, has a pull_request rule), then runs the unchanged verdict: success prints the existing success lines; a refusal prints the same ::error:: text on stderr and exits 1, but stdout now carries the listing. A detail read that fails is reported on its own row and the listing continues. The listing is a measurement: `gate`, `_release_bump` and `_require_release_rulesets` are untouched, every gate test and the existing source mutants are kept, and the probe's verdict replay over every existing ruleset fixture still compares verdict and refusal text byte for byte. The workflow release-ruleset-probe.yml is not changed. The listing reads each branch ruleset detail independently of the verdict read (two reads per detail in the probe), which docs/ci.md now says. Red-first rows: the real situation (org ruleset hidden, repository ruleset visible: both listed, the org one hidden, refusal unchanged), both hidden, one detail read failing, all visible with a valid barrier, the read-only call list, and a source mutant that drops the listing. Rewrites named for the doc-reverts check, all introduced by 9c18d41 (#488, "ci(release): read-only ruleset probe for the release gate (CR-245 F1)"): docs/ci.md "Success prints the barrier id/name and one compact inventory line with enforcement, source and bypass visibility."; docs/ci.md "Refusal exits 1 with empty stdout and the reader's `::error::` diagnostic; API/response failures name the endpoint."; docs/ci.md "Two hidden rulesets need two dispatches: the first refusal names only the first hidden ruleset; make its `bypass_actors` visible before dispatching again to identify the second." (replaced: all hidden rulesets are listed in one dispatch); the probe docstring "Read exactly the bump's ruleset evidence, with the same refusal policy."; two test comments ("The reader report's complete [] inventory fails on main without an endpoint." and "The runtime assertions already hold there; the prose does not."). The module docstring sentence "Each endpoint/page is read at most once; failures refuse without retry or sleep." (ad1ebac, #479) is scoped to the gate's reads. Lane: ci-fu-cr488m (Codex gpt-6.1-sol). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * ci(release): the probe listing bounds its inventory=ERROR row to 240 characters like the detail-error rows (CR-245 F1 measurement, sol P2) The new stdout listing printed the whole exception text on an `inventory=ERROR` row, while the per-ruleset `detail=ERROR` rows truncate it to 240 characters; a `gh` transport error with 300 characters of stderr was echoed in full. The row now applies the same bound ([:240], still !r-escaped). Four rows assert the exact bounded line (an inventory transport error, a repository transport error, a default-branch rejection with a long repository argument, an invalid summary with a long field) together with the unchanged stderr, exit code 1 and API call list of the verdict; each has a mutation control that removes only the bound and must fail. The verdict functions are untouched and the verdict's own stderr is not bounded. No documentation sentence is changed (none described a length limit). Lane: ci-fu-cr488n (Codex gpt-6.1-sol). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: ci-cx <ci-cx@invalid> Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent db58170 commit f8ec601

1 file changed

Lines changed: 107 additions & 2 deletions

File tree

‎scripts/test_ci_release_gate.py‎

Lines changed: 107 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
import re
1212
import subprocess
1313
import tempfile
14+
import unicodedata
1415
import unittest
1516
from unittest import mock
1617

@@ -2262,6 +2263,8 @@ def assert_ruleset_visibility_documentation(self, source, docs, workflow):
22622263
def constant_string(node):
22632264
if isinstance(node, ast.Constant) and isinstance(node.value, str):
22642265
return node.value
2266+
if isinstance(node, ast.Constant) and isinstance(node.value, bytes):
2267+
return node.value.decode("utf-8", errors="replace")
22652268
if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Add):
22662269
left = constant_string(node.left)
22672270
right = constant_string(node.right)
@@ -2276,8 +2279,13 @@ def constant_string(node):
22762279
if value is not None:
22772280
texts.append(value)
22782281
for text in texts:
2279-
for view in (text.replace("`", "").casefold(),
2280-
re.sub(r"""(?:#|//|['"`]|\s)+""", " ", text).casefold()):
2282+
views = (text.replace("`", ""),
2283+
re.sub(r"""(?:\\\r?\n|#|//|['"`]|\s)+""", " ", text))
2284+
# Compatibility normalization must only add matches to the original views.
2285+
casefold_views = [view.casefold() for view in views]
2286+
normalized_views = [unicodedata.normalize("NFKC", view).casefold()
2287+
for view in views]
2288+
for view in casefold_views + normalized_views:
22812289
self.assertNotRegex(view,
22822290
r"\badministration\s*:\s*read\b|\bapp\s+requires\b",
22832291
"unverified App permission claim must stay absent")
@@ -2386,6 +2394,103 @@ def test_ruleset_visibility_guard_rejects_wrapped_and_quoted_claims(self):
23862394
AssertionError, "unverified App permission claim must stay absent"):
23872395
self.assert_ruleset_visibility_documentation(*changed)
23882396

2397+
def test_ruleset_visibility_guard_rejects_backslash_wrapped_claims(self):
2398+
original = [Path(ci_release_gate.__file__).read_text(),
2399+
(ROOT / "docs/ci.md").read_text(), self.workflow()]
2400+
self.assert_ruleset_visibility_documentation(*original)
2401+
for index, claim in (
2402+
(1, "\nNeeds administration:\\\nread\n"),
2403+
(2, "\n# Needs administration:\\\n# read\n")):
2404+
with self.subTest(place=index, claim=claim):
2405+
changed = original.copy()
2406+
changed[index] += claim
2407+
self.assertNotEqual(changed[index], original[index])
2408+
with self.assertRaisesRegex(
2409+
AssertionError, "unverified App permission claim must stay absent"):
2410+
self.assert_ruleset_visibility_documentation(*changed)
2411+
2412+
def test_ruleset_visibility_guard_rejects_decoded_bytes_claims(self):
2413+
original = [Path(ci_release_gate.__file__).read_text(),
2414+
(ROOT / "docs/ci.md").read_text(), self.workflow()]
2415+
self.assert_ruleset_visibility_documentation(*original)
2416+
for expression in (
2417+
'b"administr\\x61tion: re\\x61d"',
2418+
'b"administr\\x61tion: re\\x61d".decode()',
2419+
'b"administr\\x61tion:" + b" re\\x61d"',
2420+
'b"\\xffadministr\\x61tion: re\\x61d".decode("utf-8", errors="replace")'):
2421+
with self.subTest(expression=expression):
2422+
changed = original.copy()
2423+
changed[0] += f"\nPERMISSION_CLAIM = {expression}\n"
2424+
self.assertNotEqual(changed[0], original[0])
2425+
ast.parse(changed[0])
2426+
with self.assertRaisesRegex(
2427+
AssertionError, "unverified App permission claim must stay absent"):
2428+
self.assert_ruleset_visibility_documentation(*changed)
2429+
2430+
def test_ruleset_visibility_guard_rejects_fullwidth_claims(self):
2431+
original = [Path(ci_release_gate.__file__).read_text(),
2432+
(ROOT / "docs/ci.md").read_text(), self.workflow()]
2433+
self.assert_ruleset_visibility_documentation(*original)
2434+
for index in range(len(original)):
2435+
for claim in ("\n# App requires\n", "\n# App\n# requires\n"):
2436+
with self.subTest(place=index, claim=claim):
2437+
changed = original.copy()
2438+
changed[index] += claim
2439+
self.assertNotEqual(changed[index], original[index])
2440+
with self.assertRaisesRegex(
2441+
AssertionError, "unverified App permission claim must stay absent"):
2442+
self.assert_ruleset_visibility_documentation(*changed)
2443+
2444+
def test_ruleset_visibility_guard_rejects_compatibility_suffix_claims(self):
2445+
original = [Path(ci_release_gate.__file__).read_text(),
2446+
(ROOT / "docs/ci.md").read_text(), self.workflow()]
2447+
self.assert_ruleset_visibility_documentation(*original)
2448+
for index, claim in (
2449+
(0, "\n# Needs administration: read\u2122\n"),
2450+
(1, "\nNeeds administration: read\u2122\n"),
2451+
(2, "\n# Needs administration: read\u2122\n"),
2452+
(0, "\n# Needs administration:\n# read\u2122\n"),
2453+
(1, "\nNeeds administration:\n# read\u2122\n"),
2454+
(2, "\n# Needs administration:\n# read\u2122\n")):
2455+
with self.subTest(place=index, claim=claim):
2456+
changed = original.copy()
2457+
changed[index] += claim
2458+
self.assertNotEqual(changed[index], original[index])
2459+
with self.assertRaisesRegex(
2460+
AssertionError, "unverified App permission claim must stay absent"):
2461+
self.assert_ruleset_visibility_documentation(*changed)
2462+
2463+
def test_ruleset_visibility_guard_casefold_scan_removal_breaks_restoration_controls(self):
2464+
regression = "test_ruleset_visibility_guard_rejects_compatibility_suffix_claims"
2465+
original_result = unittest.TestResult()
2466+
WorkflowPinTests(regression).run(original_result)
2467+
self.assertEqual(original_result.testsRun, 1)
2468+
self.assertEqual(original_result.errors, [])
2469+
self.assertEqual(original_result.failures, [],
2470+
"restoration controls must pass before mutation")
2471+
2472+
source = Path(__file__).read_text()
2473+
before = " for view in casefold_views + normalized_views:\n"
2474+
self.assertEqual(source.count(before), 1, "mutation must have one source target")
2475+
mutated = source.replace(before, " for view in normalized_views:\n", 1)
2476+
self.assertNotEqual(mutated, source)
2477+
guard = next(node for node in ast.walk(ast.parse(mutated))
2478+
if isinstance(node, ast.FunctionDef)
2479+
and node.name == "assert_ruleset_visibility_documentation")
2480+
namespace = {"ast": ast, "re": re, "unicodedata": unicodedata}
2481+
exec(compile(ast.Module(body=[guard], type_ignores=[]), __file__, "exec"), namespace)
2482+
case = WorkflowPinTests(regression)
2483+
case.assert_ruleset_visibility_documentation = namespace[
2484+
"assert_ruleset_visibility_documentation"].__get__(case)
2485+
result = unittest.TestResult()
2486+
case.run(result)
2487+
self.assertEqual(result.testsRun, 1)
2488+
self.assertEqual(result.errors, [])
2489+
self.assertEqual(len(result.failures), 6,
2490+
"removing the casefold-only scan must break every restoration row")
2491+
for _, traceback in result.failures:
2492+
self.assertIn("AssertionError not raised", traceback)
2493+
23892494
def test_prebuilt_builds_the_emitted_sha_and_asserts_tag_identity(self):
23902495
jobs = _jobs(self.workflow())
23912496
self.assertRegex(jobs["release"],

0 commit comments

Comments
 (0)