Skip to content

Commit 4e122b2

Browse files
Address PR #91 P1 + P2 review: per-call-site allowlist + tighten importlib.resources
P1 — Allowlist over-permits within a file. The original allowlist matched on (relative_path, surface) only, so one AllowedException("triggers.py", "attr_call:subprocess.run") blanket-permitted every subprocess.run call in triggers.py. Reviewer reproduced this: injecting a synthetic 4th subprocess.run in-memory still passed the contract test. Fix: AllowedException now pins (relative_path, surface, line, snippet). Violation carries snippet=ast.unparse(node). Match requires all four fields, so each call site needs its own entry. Adding an unreviewed subprocess.run to an already-allowlisted file now fails the test with the precise (line, surface, snippet) triple. Argv shape changes flip the snippet → also fails → forces re-review. Three subprocess.run sites in triggers.py and two in cli/discovery/artifacts.py each have distinct entries. Total ALLOWED_EXCEPTIONS grows from 7 to 13. P2 — importlib.resources.files() globally exempted. importlib.resources was in ALLOWED_FORBIDDEN_MODULE_IMPORTS, so ``files(name)`` produced no violation. Current call sites use a literal 'agents_shipgate' anchor, but a future user-controlled anchor would bypass the dynamic-import lint entirely. Fix: importlib.resources.files joins FORBIDDEN_ATTR_CALLS_EXACT, and importlib.resources joins TRACKED_NON_FORBIDDEN_MODULES so ``from importlib.resources import files; files(...)`` is resolved through name-aliases. Both call sites (triggers.py:61, fixtures.py:40) are individually pinned with snippet ``files('agents_shipgate')``. A future files(user_var) call would change the snippet and fail the test. Negative controls confirmed (both reverted before commit): - Adding ``subprocess.run(['git', 'status'], ...)`` into triggers.py at line 364 fails with "triggers.py:364 attr_call:subprocess.run snippet=...". - Replacing files('agents_shipgate') with files(some_user_anchor) in triggers.py:61 fails with snippet mismatch. New contract tests: - test_allowlist_entry_matches_real_surface — now checks all four fields (surface, line, snippet) with three failure-mode diagnostics (removed surface, line drift, snippet drift). - test_allowed_exceptions_have_no_duplicates — guards against two entries covering the same call site. - test_allowed_exceptions_pin_subprocess_run_per_call_site — regression-tests the P1 fix by asserting triggers.py has 3 distinct subprocess.run entries (not 1) and artifacts.py has 2. Scanner-internal changes: - Violation gains snippet: str (ast.unparse output). - From-import-of-forbidden-attr line now emits surface "import:<canonical>" instead of "attr_call:<canonical>" so it's distinguishable from the call-site flag in audit output. Tests: 179 lint tests pass (170 → 179); 1192 full suite passes; ruff clean; coverage 87.23%. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 5319c8c commit 4e122b2

3 files changed

Lines changed: 480 additions & 82 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,48 @@
22

33
## Unreleased
44

5+
- **v0.18 / PR #2 review follow-up: per-call-site allowlist pinning.**
6+
PR #91 review caught two structural holes in the v0.18 trust lint
7+
extension:
8+
- **P1**: the allowlist matched on `(relative_path, surface)` only,
9+
so one entry blanket-permitted every occurrence of a surface in
10+
a file. A future unreviewed `subprocess.run(...)` added to an
11+
already-allowlisted file would slip past silently.
12+
- **P2**: `importlib.resources` was globally exempted, so
13+
`files(name)` calls produced no violation. The current uses
14+
pass a literal `'agents_shipgate'` anchor, but a future
15+
user-controlled anchor would bypass the dynamic-import lint.
16+
17+
Both are closed by tightening the allowlist contract:
18+
- `AllowedException` now carries `line: int` and `snippet: str`
19+
(canonical `ast.unparse` of the offending node) in addition to
20+
`relative_path` and `surface`. `_violation_allowed` matches on
21+
all four fields. Adding a new `subprocess.run` call to an
22+
already-allowlisted file now requires a new entry; changing an
23+
existing call's argv shape changes the `snippet` and fails the
24+
contract test.
25+
- `importlib.resources.files` joins `FORBIDDEN_ATTR_CALLS_EXACT`,
26+
and `importlib.resources` joins `TRACKED_NON_FORBIDDEN_MODULES`
27+
so `from importlib.resources import files; files(...)` is
28+
resolved through name-aliases. Both call sites in `triggers.py`
29+
and `fixtures.py` are individually pinned with the literal
30+
`'agents_shipgate'` anchor in the snippet — a future
31+
`files(some_user_anchor)` call would change the snippet and
32+
fail the test.
33+
- `Violation` gains `snippet: str` captured via `ast.unparse(node)`.
34+
- New regression test
35+
`test_allowed_exceptions_pin_subprocess_run_per_call_site`
36+
asserts that multi-call files (triggers.py, artifacts.py) have
37+
distinct entries per call site, so the P1 bypass cannot
38+
reappear via consolidation.
39+
- New regression test `test_allowed_exceptions_have_no_duplicates`
40+
asserts no two entries cover the same call site.
41+
- Negative-control: injecting a 4th `subprocess.run` into
42+
`triggers.py` now fails the contract test with the precise
43+
`(line, surface, snippet)` triple. Injecting
44+
`files(user_var)` in place of `files('agents_shipgate')` fails
45+
similarly.
46+
547
- **v0.18 / PR #2 trust-hardening: static AST lint widened to entire scanner.**
648
Previously `tests/test_adapter_static_only.py` AST-scanned only
749
`src/agents_shipgate/inputs/`; the public claim in STABILITY.md and

‎STABILITY.md‎

Lines changed: 38 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -255,27 +255,46 @@ tests on every CI run, not by convention:
255255
dedicated CI step labeled *Trust-model invariant lint* before the
256256
main test suite so a regression is visible at the top of CI logs.
257257

258-
**Meta-CLI surfaces (allowlisted, audited).** Four first-party
259-
surfaces are allowed with prose rationale captured in
260-
[`tests/test_adapter_static_only.py::ALLOWED_EXCEPTIONS`](tests/test_adapter_static_only.py).
261-
Each entry must correspond to a real surface (enforced by a
262-
contract test) and must read no user code:
263-
264-
- **`cli/bootstrap.py`** — `subprocess.run([sys.executable, "-m", "agents_shipgate", …])`
265-
chains `detect → init → scan → apply-patches`. Runs only
266-
Shipgate's own CLI.
267-
- **`cli/discovery/artifacts.py`** — `subprocess.run(["git", …])`
268-
probes the user repo for file inventory. Reads git metadata only.
269-
- **`triggers.py`** — `subprocess.run(["git", "diff", …])` for trigger
270-
evaluation. Reads diff content only.
271-
- **`cli/self_check.py`** — `__import__(name)` validates that a
272-
supplied module is installed. Runs only under
258+
**Meta-CLI surfaces (allowlisted, audited).** First-party meta-CLI
259+
surfaces are pinned **per call site** in
260+
[`tests/test_adapter_static_only.py::ALLOWED_EXCEPTIONS`](tests/test_adapter_static_only.py)
261+
by a four-tuple `(relative_path, surface, line, snippet)` where
262+
`snippet` is the canonical `ast.unparse` of the offending AST node.
263+
Each entry carries a prose rationale and pins a single call:
264+
265+
- **`cli/bootstrap.py`** — one `subprocess.run` call shells the
266+
installed agents-shipgate CLI to chain
267+
`detect → init → scan → apply-patches`.
268+
- **`cli/discovery/artifacts.py`** — two `subprocess.run` calls
269+
invoke `git rev-parse` + `git ls-files` to enumerate user-repo
270+
files. Reads git metadata only.
271+
- **`triggers.py`** — three `subprocess.run` calls (`git diff
272+
--name-only`, `git diff`, `git ls-files --others
273+
--exclude-standard`) for trigger evaluation. Reads diff content
274+
only. **Plus** one `importlib.resources.files('agents_shipgate')`
275+
call to resolve the bundled trigger catalog.
276+
- **`fixtures.py`** — one `importlib.resources.files('agents_shipgate')`
277+
call to resolve the bundled fixture directory.
278+
- **`cli/self_check.py`** — one `__import__(module_name)` call
279+
validates that supplied modules import cleanly. Runs only under
273280
`agents-shipgate self-check`, never during scan.
274281

275-
A contract test pins each allowlist entry to a real surface and
276-
fails if the entry goes stale; a second contract test sweeps the
277-
whole scanner package and fails on any forbidden surface that lacks
278-
an allowlist entry.
282+
Per-call-site pinning means **adding a second occurrence of an
283+
already-allowlisted surface in the same file STILL requires a new
284+
entry**. Changing the call's argv shape (the `snippet` changes)
285+
also fails the test, forcing a reviewer to confirm the change is
286+
benign. The literal-anchor invariant for
287+
`importlib.resources.files('agents_shipgate')` is enforced by
288+
snippet pinning: a future `files(user_var)` call would not match.
289+
290+
Three contract tests pin the audit trail:
291+
`test_allowlist_entry_matches_real_surface` (every entry matches a
292+
real violation on all four fields),
293+
`test_no_unallowlisted_forbidden_surface_in_scanner` (every
294+
observed violation has a matching entry), and
295+
`test_allowed_exceptions_pin_subprocess_run_per_call_site` (the
296+
multi-call files have distinct entries per call site, regression-
297+
testing the structural fix from the v0.18 PR #2 review).
279298
- **[`tests/test_fixture_no_import.py`](tests/test_fixture_no_import.py)** —
280299
per-adapter live-load tests. Each adapter (LangChain, CrewAI, OpenAI Agents
281300
SDK, Google ADK, MCP, OpenAPI, Anthropic, OpenAI API, n8n, Codex plugin) is

0 commit comments

Comments
 (0)