Skip to content

Commit 3062ea7

Browse files
sahrizviclaude
andauthored
review: --no-ai fix + --explain-tier / --force-tier flags + manifest auto-discovery + column-aware schema.yml test-removal (#1027)
* feat(review): observability + recall (Round 18, B1 + G1 + G2 + G3 + G6) Round-17 bench established three baselines: - A1-default: 7/64 recall (11%), 4% precision - A1-workaround (with --manifest): 3/8 on applicable jaffle scenarios - Baseline B: 0/64 (plugin never invoked the CLI — separately fixed) This ships the six review-side changes documented as improvement paths in the Round-18 plan. Each is guarded to avoid regressing the current customer numbers. ## B1 — bare `--no-ai` triggers yargs help path yargs' automatic `--no-<option>` negation collides with the option literally named `no-ai`: bare `--no-ai` is interpreted as "set undeclared option `ai` to false" and the command silently falls to the help path (exit 0, no review runs). Every `--no-ai` invocation across the bench was affected — our earlier A1 numbers were captured with `--no-ai --yolo` but the flag was inert and the AI lane always ran. Fix: `.parserConfiguration({ "boolean-negation": false })` on the review command's yargs builder. `--no-ai` now binds to the declared `noAi` flag as authored. `--no-ai=true` / `--no-ai=false` still work for programmatic parity. ## G1 — `--explain-tier` flag Adds a boolean flag that surfaces the tier classifier's reason list on the verdict envelope (`tierReasons: string[]`) and in the human- readable render (`> 🧭 **Tier: X** — ...`). Read-only — doesn't change tier classification or verdicts. When the flag is off, the envelope omits `tierReasons` entirely (backwards compatible; existing consumers unaffected). ## G2 — `--force-tier <trivial|lite|full>` flag EXPERIMENTAL / bench-debug only. Overrides the classifier's tier so we can measure the tier-gate contribution to recall misses (e.g. js2 gets `trivial/0` — can't tell if the catalog missed vs. the tier gate filtered it out). Guardrails: - Prints an "EXPERIMENTAL (bench / debug only)" warning to stderr every use. - Envelope carries `tierForced: true` and `tierClassified: <original>` so audits can see the bypass. - `tierReasons` is force-populated with a leading marker even if `--explain-tier` isn't set, so the renderer always shows the tier was forced. - Verdict header renders as `full tier — forced (was trivial)`. Not a default-visible feature. Documented as debug-only in the yargs `describe`, and the stderr banner ensures no customer accidentally uses it in CI without noticing. ## G3 — manifest auto-discovery + freshness warning Before: `--manifest <path>` had to be passed explicitly. Otherwise the CLI used the config-relative default `target/manifest.json`, which silently missed whenever `cwd` wasn't exactly the dbt project root — every such review degraded to lint-only. After (in `run.ts`): 1. If `--manifest` isn't explicit AND the config-relative path doesn't exist, walk UP from `cwd` looking for `dbt_project.yml`. Use the adjacent `target/manifest.json` when it exists. 2. Log the discovery to stderr (`ℹ️ auto-discovered dbt manifest at ...`) so customers see which manifest the review used. 3. Refuse to auto-discover a manifest from a directory that doesn't contain a dbt project — a `target/` from an unrelated tool (e.g. Airflow) never gets picked up. 4. Freshness warning: when `--head` is set (CI / bench shape) AND any changed file has an mtime newer than the manifest, print a `⚠️ manifest ... appears stale` message to stderr. Skipped for working-tree diffs because mtime noise during active edits would spam warnings. Explicit `--manifest <path>` always wins. Auto-discovery is only attempted when the caller was silent AND the config default is absent. ## G6 — column-aware schema.yml test-removal detector Before: the detector compared removed vs added test lines as bare strings (`- unique`, `- not_null`). Any sibling column that STILL had the same test type re-added its line under a different indent (yaml re-serialization), silently cancelling the genuine removal from a different column. js2 (drops `unique` + `not_null` from `customers.customer_id` while `orders.order_id` still has them) hit this exact bug: 0/2 recall in both A1 modes across Round 17. After: `collectTestOccurrencesFromDiff` walks the unified diff tracking `(model, column)` context via nearest preceding `- name: <X>` headers and hunk boundaries. Removed tests are keyed by `(model, column, test)` and only cancelled by a re-add on the SAME tuple. One finding per removal (not one summary), with severity elevated to `warning` for `unique` removals and for `not_null` on `_id` / `_key` / `id` columns (silent-PK-null risk). Verified on js2 locally: 0 findings → 2 warnings. ## Verdict envelope schema `VerdictEnvelope` gains three optional fields, all included in the signed canonical body so tampering is detectable: - `tierReasons?: string[]` — from G1 (surfaces classifier reasons) - `tierForced?: boolean` — from G2 - `tierClassified?: RiskTier` — from G2 (original tier before force) All are optional and absent in the default (no-flag) case, so existing verdicts remain byte-equivalent to before. ## What's NOT in this change - G4 (`verify` subcommand) — deferred; doesn't move measured metrics. - G5 (`--commit` / `--pr` / `--files`) — deferred; adoption UX only. - G7 (severity mapping tune-up) — deferred; needs its own design pass. - Real-MR taxonomy expansion — larger design lift; addressed in a later round. ## Follow-up Rerun A1-default and A1-workaround on the 13-scenario corpus with these changes and publish the deltas as Round 18 in the reviewer plugin journal. * fix(review): Codex R18 review follow-ups — G2 audit guardrail + G6 copy Independent code review by Codex (2026-07-21) surfaced two issues in the R18 fixes: 1. G2 guardrail bug — `tierForced` was only set to `true` when the forced tier value happened to DIFFER from the classifier's decision. So `altimate-code review --force-tier=full` on a PR the classifier would naturally rate `full` still runs the debug bypass (silently exercises the flag path in orchestrate.ts) but emits an envelope with no `tierForced`, no `tierClassified`, and no "forced via" reason — breaking the audit-trail invariant we documented in the R18 commit. Fix: `tierForced = input.forceTier !== undefined`. Whenever the caller passed the flag, the envelope now records the bypass regardless of whether the forced value matched the natural one. The leading reason string now also names the forced value. Verified on js4 (naturally `full`): `--force-tier=full` now produces `tierForced: true, tierClassified: "full", tierReasons: ["forced via --force-tier=full (classifier said full)", ...]`. 2. G6 cosmetic copy issue — the schema.yml test-removal detector's body claimed "Removing a `unique` test on a mart-layer key is how silent duplicate rows ship" for EVERY unique-test removal, even when the schema.yml lives outside a mart layer (e.g. js2's `models/schema.yml` sits at the top level, not under `models/marts/`). Fix: detect the layer from the file path (`marts?` or `reporting` → "mart-layer"; otherwise "declared") and interpolate that into the body. The finding is still accurate for genuine mart-layer removals; the copy no longer misattributes for staging or top-level schema files. Neither change alters what findings are surfaced, only the audit envelope shape (G2) and finding body prose (G6). No test-file changes; smoke tests on js2 + js4 confirm both. * fix(review): address PR-1027 code-review feedback (blocker + majors + minors) Independent multi-reviewer code review on the PR surfaced one blocker, two majors, and three minors. This commit addresses each finding and adds regression-guard tests. ## BLOCKER — column-aware detector regresses on real diffs + broke its own test (finding #1) The R18 `collectTestOccurrencesFromDiff` walker required both `- name: model` AND `- name: column` inside the same diff hunk to attribute a removed test. With git's default `-U3` context the model header sits many lines above a column's `tests:` block and isn't in the hunk — so the walker misclassified the first `- name:` it saw as a model, never set `currentColumn`, and silently dropped the removal. The existing unit test at test/altimate/review-dbt-patterns.test.ts:141 (`"- - unique\n- - not_null"`) went from pass to fail under R18, so the branch was shipping a red test suite as well. Rewrite `detectSchemaYmlPatterns` to prefer STRUCTURAL YAML diffing: - Accept optional old/new file content via a `SchemaYmlDetectContent` parameter. When provided, parse both sides with the `yaml` package, walk `models[]`, `snapshots[]`, `sources[]`, `seeds[]`, extract every `(entity, column?, test)` tuple, diff old vs new sets, emit one finding per genuine removal. - Supports both column-level and model-level tests (dbt allows `tests:` / `data_tests:` directly under the entity for surrogate-key `unique`, etc.) — closes MAJOR finding #2. - Handles both `tests` and `data_tests` (dbt 1.8+ alias), both bare (`- unique`) and block-form (`- relationships: {...}`) tests, and quoted / commented YAML names — closes MINOR finding #6. - Sources are qualified as `source.table.column` in the model field. Fall back to the pre-R18 string-based line detection when no content is provided (e.g. unit-test callers, offline CI diffs without a content resolver). Fallback emits a suggestion / warning without column attribution — cannot distinguish "moved to sibling column" from "removed" with diff-only input, that limitation is inherent. The existing test's expectation is preserved. Wire the orchestrator to always supply old/new content when calling the detector on modified schema.yml files, so the fallback only fires in tests / diff-only CI paths, not in production reviews. ## MAJOR — auto-manifest projectRoot not threaded to compiled resolver (finding #3) `autoDiscoverManifest` returned `{ path, projectRoot }` and rebased `manifestAbs` correctly, but `dbtProjectName(opts.cwd)` and `makeCompiledResolver({ cwd: opts.cwd })` still used `opts.cwd`. When the CLI was invoked from a subdirectory, auto-discovery would find the manifest in an ancestor project but the compiled resolver looked for `target/compiled/…` under the subdir and never found it — engine lanes silently fell back to raw Jinja. Thread a `dbtRoot` variable through the auto-discovery path: starts as `opts.cwd`, updated to `discovered.projectRoot` when G3 fires. Use it for both `dbtProjectName(dbtRoot)` and `makeCompiledResolver({ cwd: dbtRoot })` so compiled SQL is found next to the discovered manifest. ## MINORS - **#4 docstring** — `autoDiscoverManifest`'s docstring cited defenses ("relative escapes", "under project's parent") that aren't in the code. Rewrote the doc to describe what actually happens: walk up for `dbt_project.yml`; return the adjacent `target/manifest.json` when present; never grab a `target/` from an unrelated tool that happens to sit above us on the filesystem. - **#5a verdictHeadline undefined** — `env.tierClassified` is optional in the schema; an externally-built envelope with `tierForced: true` but no `tierClassified` would render `was undefined`. Added a `?? "unknown"` fallback in `format.ts`. - **#5b unbounded tierReasons in summary** — when `--force-tier` is passed on a large PR, `tierReasons` prepends the forced marker AND spreads all `classifyPR().reasons` (one entry per file that forces the FULL tier). Rendered comment was bloating. Cap the rendered summary at the first 8 reasons with a "+N more in verdict envelope" overflow marker; the full list stays in the signed envelope for audit. ## Tests Added 9 new regression tests: - `dbt-patterns` — structural: sibling-column edge case (the exact case G6 claims to fix), model-level test removal with attribution metadata, block-form `- relationships:`, `data_tests` alias, source-column tests, added-file returns 0 removals, fallback diff-only path preserves the existing test's expectation. - `verdict` — `--force-tier` envelope audit: `tierForced` and `tierClassified` are set whenever the flag is passed (including when forced tier matches classifier), and are covered by the HMAC signature (tampered envelope stripping the audit fields fails verifyEnvelope). Test suite: 155/155 passing across the six review test files (previously 46/47 — the one test the R18 branch broke is back green). Typecheck clean. * fix(review): address PR-1027 bot-review round-2 (dedupe collapse, rename bypass, snapshot yml, +5 more) After the earlier fixes on `b32065b23`, three independent reviewer bots (coderabbitai, cubic-dev-ai, dev-punia-altimate) surfaced 8 more issues on the schema.yml test-removal path — 2 HIGH, 1 MAJOR, 4 MEDIUM, 1 LOW — plus an independent local review round after that caught an additional blocker downstream of the fixes. This commit addresses each. ## BLOCKER — fallback distinct-removal fix defeated by global dedupe The R18-follow-up fix removed a local dedup in the fallback path so each removed test-line produced its own detector-level finding. But `runReview` runs a global `dedupe(merged)` step that fingerprints findings by (category, file, model, column, ruleKey) via `finding.ts:107`. For fallback findings sharing `file`, empty `model`, empty `column`, and a ruleKey varying only by `test`, two `unique` removals in the same file still collapsed to one downstream. The detector-level test at `review-dbt-patterns.test.ts:397` didn't catch it because it asserts against the pre-dedupe output. Fix: give fallback findings a per-diff occurrence-index discriminator appended to the ruleKey (`.#0`, `.#1`, …) so distinct removals get distinct fingerprints. Structural (attributed) findings unchanged — `(model.column.test)` is already unique per removal. Added an integration-shape regression test at `review-dbt-patterns.test.ts:432` that pipes detector output through `dedupe(f)` and asserts 4 distinct ids survive for 4 distinct removals. Also a same-shape guard for the structural path. ## HIGH — renamed schema.yml bypasses detector The orchestrator gated `oldContent` fetch on `file.status === "modified"`. A schema.yml being renamed (e.g. moved to a new subdir) that also dropped a `unique`/`not_null`/`relationships` guardrail silently skipped the detector because `oldContent` came back undefined and the structural path treated the file as newly-added. Fix: gate on `file.status !== "added"` so renamed files fetch the old side; separately, changed the detector so an undefined `oldContent` on a modified/renamed file falls through to the diff-based fallback (does not silently treat as added-file). Added test at `review-dbt-patterns.test.ts:332` covering the rename shape. ## MAJOR — structural path treats `oldContent=undefined` as added file `canUseStructural = newContent !== undefined && (isAddedFile || oldContent !== undefined)`. When the content resolver returns undefined on a modified/renamed file for a transient reason (git failure, ref not readable), we now leave `usedStructural = false` and let the fallback surface the raw diff removals — instead of producing an empty structural-diff and silently dropping every removal in the diff. ## MEDIUM — snapshot YAML property files never reached the detector `classifyDbtFile` matched everything under `snapshots/` as `snapshot` regardless of extension, so `snapshots/*.yml` never reached the `schema_yml` gate in the orchestrator. The new `snapshots:` branch added to the structural walker was dead code for real snapshot property files. Fix: split the classification by extension. `snapshots/*.sql` remains `snapshot` (unchanged tier-forcing / catalog rules). `snapshots/*.yml` classifies as `schema_yml` (routes to the test-removal detector). Added `classifyDbtFile` assertions for both extensions and an end-to-end structural-detector test on a snapshot yml. ## MEDIUM — redundant per-model summary line `isFirstForModel` fired once per distinct model, so a diff touching two models produced two copies of the same aggregate summary ("This PR removes N tests total on model(s) A, B") appended to separate findings. Fix: replaced with a single file-scoped `summaryEmitted` boolean. The aggregate summary is appended to the first finding for the file, once, and the distinct-models list is computed from the whole removals array. Test at `review-dbt-patterns.test.ts:454` asserts exactly one finding carries the aggregate line when two models have removals. ## MEDIUM — `warnIfStale` fired on unrelated file changes `warnIfStale` compared mtimes for every changed path. Any `README.md` / `package.json` / `.github/*` change post-manifest triggered a false-positive stale warning. Fix: introduced `isManifestAffecting()` — admits only `.sql|.py|.yml|.yaml|.csv|.md` under `models|seeds|snapshots|macros|tests|analyses/`, plus root `dbt_project.yml|packages.yml|profiles.yml|dependencies.yml`. Explicitly admits `.md` under `models/` because dbt docs blocks live there. Exported for tests; added a table-driven test file `review-run-stale.test.ts` covering both admitted and rejected cases (README.md, models/foo.sql, docs/foo.md, models/foo.md, seeds/lookup.csv, snapshots/*.sql, snapshots/*.yml, dbt_project.yml, .github/workflows/, target/…). ## LOW — YAML parse errors silently ignored Both `YAML.parse` failures (new content, old content) now log via `Log.create({ service: "review", tag: "detectSchemaYmlPatterns" })` with the file path and error message, then fall through to the diff-based fallback. Not fail-open. ## PERF — sequential schema.yml `getContent` calls The orchestrator's schema.yml loop was serial `await` per file; schema-heavy PRs paid one round-trip per file. Refactored to `Promise.all` across the file list with per-file `Promise.all` for old/new. Ordering preserved by `Promise.all` contract. ## Test status 188 pass / 0 fail across the 7 review test files. Typecheck clean. End-to-end sanity on the js2 test-removal scenario: CLI still surfaces both `unique` and `not_null` warnings, verdict `COMMENT/trivial/2`, envelope signature verifies. * perf(review): address PR-1027 round-3 nits (deleted-file fetch, summary alloc, shift O(N²)) Three centralized-review findings on the previous commit `6729f5374`, all perf/hygiene: ## MEDIUM — schema.yml orchestrator fetched `oldContent` for deleted files `file.status !== "added"` on the gate matched `"deleted"` too, triggering a `git show` round-trip whose result the detector immediately discarded (early-returns `[]` for `status === "deleted"`). Fix: filter deleted schema files out of the loop upfront — `reviewable.filter((f) => f.kind === "schema_yml" && f.status !== "deleted")`. The `newContent` fetch also simplifies to unconditional `getContent?.(file.path, "new")` since deleted files never reach the loop. ## MEDIUM — `distinctModels`/`modelClause` recomputed on every loop iteration The `[...new Set(...).filter(...).map(...).join(...)]` chain was evaluated on every removal even though the result is only used when `shouldEmitSummary` is true (once per file). Moved the computation inside the guard. ## LOW — `fallbackDiscriminators.shift()` inside the loop is O(N²) `Array.prototype.shift()` re-indexes all remaining elements on each call. Replaced with a hoisted `let fallbackIdx = 0` + `fallbackDiscriminators[fallbackIdx++]`. Negligible on 2-3 removals but grows quadratically on wider PRs. ## Test status 188 pass / 0 fail across the 7 review test files. Typecheck clean. No behavior change beyond the deleted-file fetch skip (which is a pure performance improvement — same output, fewer round-trips). * fix(review): address PR-1027 round-4 (deleted schema.yml + backtick fence + Bun-alias flake + finding attribution) Four items from cubic-review round-3 + one CI-only test flake: - **Deleted schema.yml handling (cubic P2)** — `detectSchemaYmlPatterns` now diffs old YAML against `{}` when `status === "deleted"`, so dropping a whole schema file surfaces every prior test as a removal. `orchestrate.ts` no longer filters deleted files and skips `newContent` fetch for them (git-show at HEAD would fail). Two regression tests: 4-test deletion, and safe-degrade when `oldContent` is missing. - **Backtick-safe tierReasons rendering (cubic P3)** — `format.ts` now sizes the inline-code-span fence to `max(backtick-run) + 1` and pads leading/trailing backticks with a space, so paths like `` `foo`bar` `` can't terminate the span or inject Markdown. Regression test covers single/double/leading/trailing/both-ends cases. - **Structured attribution on removal findings (cubic P3)** — `removed_tests` findings now include `model` and `column` on the top-level Finding, not just inside `evidence.result`, so downstream dedupe / formatting / telemetry see them. - **TUI network-options test flake (CI-only)** — Bun's transpiler statically resolved the `@/cli/network` string literal in `toContain('...')` into a `file:///…/network.ts` URL, inverting the comparison. Rewritten as `toMatch(/regex/)` so the alias literal isn't recognizable to Bun's analyzer. Additional edge tests per codex round-4 review: - Deleted schema.yml with unparsable oldContent → diff-only fallback. - Deleted empty schema.yml → no fabricated findings. - tierReasons with both leading + trailing backticks → symmetric padding. Tests: 145 pass, 0 fail in touched suites; full review suite (188 tests) green. Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * fix(review): address consensus-review findings round-5 (1 MAJOR + 4 MINOR + 1 NIT + 2 missing tests) Addresses the panel's findings on PR #1027. Summary of what landed: - MAJOR #1 — subdir-invocation content-resolution: `makeContentResolver` working-tree read, `warnIfStale` file existence, and `makeCompiledResolver` compiled-SQL lookup all previously joined repo-relative paths from `git diff --name-status` with the caller's `opts.cwd`, silently ENOENT-ing when the CLI was invoked from a subdir. Now `reviewPullRequest` resolves `git rev-parse --show-toplevel` once via a new `gitRepoRoot()` helper, passes it to `makeContentResolver` (new `gitRoot` opt) and `warnIfStale` (renamed to `fsRoot`), and computes `pathPrefix = path.relative(gitRoot, dbtRoot)` for `makeCompiledResolver`. 8 new regression tests in `review-subdir-invocation.test.ts`; codex-round-5 HIGH on Windows path separator normalization also addressed (pathPrefix is normalised to POSIX before matching git-diff paths). *Note on authorship*: the `makeContentResolver` working-tree read (`fs.readFile(path.join(opts.cwd, file))` in `git.ts`) is pre-existing code from May 2026 (anandgupta42's commit `79c9ddd22`), not introduced in this PR. The subdir-invocation bug it triggers surfaces via the R18 work in this PR (`warnIfStale` + `makeCompiledResolver` on `dbtRoot`), which is why the consensus panel scoped it here. Fix is defensive tightening — makes the flow correct end-to-end. - MINOR #2 — envelope zod invariant: `VerdictEnvelope` now carries a `superRefine` enforcing (`tierForced=true` ↔ `tierClassified` defined), and `tierForced: false` is rejected as illegitimate (the marker exists to positively record a bypass; unforced runs must omit it entirely). 4-case regression test. - MINOR #3 — per-file "N data tests" summary: reworded from ambiguous "This PR removes N data tests in total" to schema-file-scoped "This schema file drops N data tests" (per PR #1027 consensus). Existing regression test updated. - MINOR #4 — multiple `relationships` tests on one column collapsed: `extractTestOccurrences` now discriminates `relationships` by (to, field) via a `\x01<to>\x02<field>` suffix on the occurrence key. Downstream, the discriminator is stripped from display copy (title/ body) but retained in `ruleKey` so the global fingerprint dedupe keeps distinct relationships-on-same-column findings separate. Regression tests cover both-removed (→ 2 findings) and one-removed (→ 1 finding). Codex-round-5 minor on colon-collision in the tag also addressed by preserving the internal `\x02` separator inside `ruleKey` rather than colon-joining (avoids `to='a:b'`/`field='c'` collision). - MINOR #5 — `boolean-negation: false` regression coverage: new parametric test in `review-ci.test.ts` iterating declared boolean flags (`json`, `post`, `no-ai`, `explain-tier`) and asserting each parses to the expected type + default under bare / `=true` / `=false` invocations. Locks the parser configuration against future breakage. - NIT #6 — `autoDiscoverManifest` loop tidy: rewritten as a `for`-loop using the `path.dirname(dir) === dir` fixed-point check, removing the earlier redundant `if (dir === root)` guard. Not addressed: - NIT #7 — positional fallback discriminator was noted stable within a diff by the reviewer; no code action required. Full altimate review suite: 3787 pass / 640 skip / 0 fail (133 files; up from 3773 baseline — 14 new tests). Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * fix(review): address post-round-5 bot-review findings on PR #1027 Five items from coderabbit / cubic-dev-ai / kilo-code-bot on round 5: - Symlink-escape containment (coderabbit + cubic): `makeContentResolver` working-tree read and `makeCompiledResolver` compiled-SQL read now go through a `realpath` containment check that rejects any target whose resolved path falls outside the resolved root. A tracked symlink like `models/evil.sql → /etc/passwd` no longer leaks external content into the review pipeline. Applied at both the diff-side (git.ts) and compiled-side (compiled.ts) resolvers. - Symlink-consistent path prefix (cubic + kilo): `path.relative(gitRoot, dbtRoot)` produced a bogus climb path (`../../../var/...`) whenever `dbtRoot` traversed an unresolved symlink (macOS `/var` → `/private/var` is the canonical case). The mapped prefix never matched incoming repo-relative paths, so compiled SQL was silently missed on the exact monorepo layouts the subdir fix was meant to enable. Both roots are now realpath-resolved before `path.relative`, and `dbtRootReal` is handed to `makeCompiledResolver` as its `cwd`. Falls back gracefully when realpath fails (path deleted mid-run). - `gitRepoRoot` newline handling (cubic P3): `.trim()` also stripped legitimate leading/trailing path whitespace. Replaced with an explicit `\r?\n` terminator strip that leaves everything else intact. - dbt version comment (cubic P3): the `arguments:` nesting syntax is documented from dbt v1.10.5, not 1.9+. Comment updated. New regression tests in `review-subdir-invocation.test.ts`: - makeContentResolver refuses to read a symlink escaping the git root - makeCompiledResolver refuses to read a compiled symlink escaping the compiled root - gitRepoRoot preserves legitimate path characters (no trim() regression) Full altimate review suite: 3790 pass / 640 skip / 0 fail (was 3787 baseline — 3 new tests, +8 expect calls). Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * refactor(review): [PR #1027] share safeReadInside between git.ts and compiled.ts cubic + kilo bot review — the realpath containment check was duplicated across `makeContentResolver` (git.ts) and `makeCompiledResolver` (compiled.ts). Both bots flagged that keeping two copies of security- sensitive logic risks a future fix (e.g. Windows case-sensitivity, trailing-separator edge) landing in one call site but not the other. - Export `safeReadInside(root, rel)` from git.ts as the single realpath-checked reader. - `compiled.ts` imports it and replaces its inline containment block with `return await safeReadInside(compiledRoot, rel)`. - No behavior change; regression suite still 3790 pass / 0 fail. Co-Authored-By: Claude Opus 4.7 <[email protected]> Claude-Session: https://claude.ai/code/session_017zXDXMiNFh4qDPxPCfa2of * fix(review): address altimate-harness-bot review findings on run.ts Three findings from the harness-bot pass on PR #1027: 1. `run.ts:114` — add rationale for the fixed-point walk's early return. When `dbt_project.yml` exists but `target/manifest.json` doesn't, we intentionally return `undefined` rather than continue walking to a grandparent's compiled project — a grandparent's DAG is a different project's DAG. Comment makes this clear so future maintainers don't "fix" it into a silent grandparent-fallback. 2. `run.ts:129` — narrow the `.md` match in `isManifestAffecting`. dbt `{% docs %}` blocks are canonically parsed under `models/` and `analyses/`. Under `macros/`, `seeds/`, `snapshots/`, and `tests/`, `.md` files are package documentation (READMEs), not manifest input. Split the check so `.md` requires `models|analyses`; other extensions (`.sql`, `.py`, `.yml`, `.yaml`, `.csv`) still admitted under all six source directories. Added `README.md` under each of the four newly-excluded directories to the rejected-path test list. 3. `run.ts:120-124` — JSDoc block that describes `warnIfStale` was physically placed above `isManifestAffecting`, so TS/IDE tooling would attribute both blocks to the earlier symbol and `warnIfStale` would show no hover doc. Split into two blocks: a self-contained docstring for `isManifestAffecting` (its own concerns + why it's exported) and a fresh docstring for `warnIfStale` immediately preceding its definition. Tests: 210/210 green (7 review-* files); review-run-stale.test.ts gains `analyses/gross_margin.md` (admitted) plus four `.md`-under- non-docs-dirs entries in the rejected list. --------- Co-authored-by: Haider <[email protected]> Co-authored-by: Claude Opus 4.7 <[email protected]>
1 parent 330c950 commit 3062ea7

15 files changed

Lines changed: 2004 additions & 56 deletions

‎packages/opencode/src/altimate/review/compiled.ts‎

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { promises as fs } from "node:fs"
22
import path from "node:path"
33
import YAML from "yaml"
4+
import { safeReadInside } from "./git"
45

56
/**
67
* Resolve dbt's COMPILED SQL for static analysis.
@@ -36,12 +37,20 @@ export async function dbtProjectName(cwd: string): Promise<string | undefined> {
3637
}
3738

3839
export interface CompiledResolverOptions {
40+
/** dbt project root (the dir containing `dbt_project.yml`). */
3941
cwd: string
4042
projectName?: string
4143
/** Directory holding HEAD-side compiled SQL (relative to cwd). */
4244
headDir?: string
4345
/** Directory holding BASE-side compiled SQL (relative to cwd). */
4446
baseDir?: string
47+
/** Prefix within the repo-relative file path that maps to `cwd`.
48+
* For a monorepo where the dbt project lives at `packages/dbt/`, callers
49+
* pass `pathPrefix: "packages/dbt"` so a repo-relative path like
50+
* `packages/dbt/models/foo.sql` resolves inside the dbt project root as
51+
* `models/foo.sql` before joining with `target/compiled/<project>/`.
52+
* Omit when `cwd` IS the repo root. */
53+
pathPrefix?: string
4554
}
4655

4756
/**
@@ -52,15 +61,29 @@ export function makeCompiledResolver(opts: CompiledResolverOptions) {
5261
const project = opts.projectName
5362
const headDir = opts.headDir ?? "target/compiled"
5463
const baseDir = opts.baseDir ?? "target-base/compiled"
64+
// Normalise the prefix so both "" and "." mean "no prefix". Match against
65+
// git-style forward slashes because `git diff --name-status` always emits
66+
// POSIX separators regardless of platform. `path.relative()` on Windows
67+
// returns backslashes, so callers passing that verbatim would produce a
68+
// prefix that never matches — normalise here (codex R20 review HIGH).
69+
const prefix =
70+
opts.pathPrefix && opts.pathPrefix !== "."
71+
? opts.pathPrefix.replace(/\\/g, "/").replace(/\/+$/, "")
72+
: ""
5573

5674
return async (file: string, side: "old" | "new"): Promise<string | undefined> => {
5775
if (!project) return undefined
76+
// When the dbt project sits inside a subdir of the repo, `file` (from
77+
// `git diff --name-status`) is repo-root relative and always uses
78+
// POSIX separators. Strip the mapped prefix so it becomes dbt-root
79+
// relative before joining with `cwd` (which IS the dbt root). Without
80+
// this the compiled resolver silently misses in monorepo layouts.
81+
const rel = prefix && (file === prefix || file.startsWith(prefix + "/")) ? file.slice(prefix.length + 1) : file
5882
const root = side === "new" ? headDir : baseDir
59-
const full = path.join(opts.cwd, root, project, file)
60-
try {
61-
return await fs.readFile(full, "utf8")
62-
} catch {
63-
return undefined
64-
}
83+
const compiledRoot = path.join(opts.cwd, root, project)
84+
// Shared realpath containment check — matches makeContentResolver's
85+
// symlink-safe read so a future tweak to the containment logic can't
86+
// leave one call site behind (cubic + kilo suggestion).
87+
return await safeReadInside(compiledRoot, rel)
6588
}
6689
}

‎packages/opencode/src/altimate/review/dbt-patterns.ts‎

Lines changed: 430 additions & 28 deletions
Large diffs are not rendered by default.

‎packages/opencode/src/altimate/review/diff-filter.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,11 @@ export function classifyDbtFile(path: string): DbtFileKind {
6161
const isYaml = p.endsWith(".yml") || p.endsWith(".yaml")
6262
if (/(^|\/)(dbt_project|profiles|packages|dependencies)\.ya?ml$/.test(p)) return "project_config"
6363
if (/(^|\/)macros\//.test(p)) return "macro"
64-
if (/(^|\/)snapshots\//.test(p)) return "snapshot"
64+
// Snapshots split by extension: `.sql` files are snapshot definitions
65+
// (tier-forcing, catalog-scoped), YAML property files under `snapshots/`
66+
// are schema files (declared tests + column metadata) and should route to
67+
// `schema_yml` so the test-removal detector runs on them.
68+
if (/(^|\/)snapshots\//.test(p) && p.endsWith(".sql")) return "snapshot"
6569
if (/(^|\/)seeds\//.test(p) && p.endsWith(".csv")) return "seed"
6670
if (/(^|\/)tests\//.test(p) && p.endsWith(".sql")) return "test"
6771
if (/(^|\/)analyses\//.test(p)) return "analysis"

‎packages/opencode/src/altimate/review/format.ts‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,12 @@ export function verdictHeadline(env: VerdictEnvelope): string {
2828
[critical && `${critical} critical`, warning && `${warning} warning`, suggestion && `${suggestion} suggestion`]
2929
.filter(Boolean)
3030
.join(", ") || "no findings"
31-
return `${VERDICT_LABEL[env.verdict]} — ${counts} (${env.tier} tier)`
31+
// tierClassified is optional in the schema — guard against externally-built
32+
// envelopes that mark tierForced without threading the original classification.
33+
const tierLabel = env.tierForced
34+
? `${env.tier} tier — forced (was ${env.tierClassified ?? "unknown"})`
35+
: `${env.tier} tier`
36+
return `${VERDICT_LABEL[env.verdict]} — ${counts} (${tierLabel})`
3237
}
3338

3439
/** Full PR/MR summary comment body (markdown), prefixed with the dedup marker. */
@@ -43,6 +48,37 @@ export function renderSummary(env: VerdictEnvelope): string {
4348
)
4449
}
4550

51+
// G1 (Round 18) — surface classifier reasons when --explain-tier populated
52+
// them, so a customer can see why the review ran at this tier. Also surfaces
53+
// when --force-tier bypassed the classifier (tierReasons is auto-populated).
54+
// Truncate the rendered summary on very large diffs: classifyPR appends one
55+
// reason per file forcing FULL tier (e.g. every touched schema.yml or every
56+
// PII column), which bloats the comment on wide PRs. The full list stays in
57+
// the signed envelope's tierReasons[]; the summary shows the first 8.
58+
if (env.tierReasons && env.tierReasons.length) {
59+
const RENDER_CAP = 8
60+
// Pick an inline-code-span fence longer than any backtick run inside `r`
61+
// so a path like `packages/…/foo`bar`.sql` cannot terminate the span
62+
// (cubic-review P3).
63+
const shown = env.tierReasons
64+
.slice(0, RENDER_CAP)
65+
.map((r) => {
66+
const runs = r.match(/`+/g)
67+
const maxRun = runs ? Math.max(...runs.map((run) => run.length)) : 0
68+
const fence = "`".repeat(maxRun + 1)
69+
// If the reason itself starts/ends with a backtick, pad with a space so
70+
// the leading/trailing backtick isn't glued to the fence.
71+
const pad = /^`|`$/.test(r) ? " " : ""
72+
return `${fence}${pad}${r}${pad}${fence}`
73+
})
74+
.join(", ")
75+
const overflow =
76+
env.tierReasons.length > RENDER_CAP
77+
? ` (+${env.tierReasons.length - RENDER_CAP} more in verdict envelope)`
78+
: ""
79+
lines.push(`> 🧭 **Tier: ${env.tier}** — ${shown}${overflow}`, "")
80+
}
81+
4682
if (!env.findings.length) {
4783
lines.push("No issues found in the changed dbt models. 🎉", "")
4884
} else {

‎packages/opencode/src/altimate/review/git.ts‎

Lines changed: 60 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,42 @@ export async function collectChangedFiles(opts: CollectOptions): Promise<Changed
6262
)
6363
}
6464

65+
/** Read the given file only when its realpath sits inside the resolved root.
66+
* Blocks the "tracked symlink escapes the root" class of attack — e.g. a
67+
* `models/evil.sql → /etc/passwd` symlink that would otherwise leak external
68+
* files into the review pipeline (coderabbit + cubic security review).
69+
* Returns undefined when the target escapes, is missing, or realpath fails.
70+
*
71+
* Exported for the compiled-SQL resolver in compiled.ts so both content
72+
* paths go through the same containment check — cubic + kilo suggested
73+
* extracting the shared helper so a future security tweak (Windows
74+
* case-sensitivity, trailing separator) can't leave one call site
75+
* behind. */
76+
export async function safeReadInside(root: string, rel: string): Promise<string | undefined> {
77+
try {
78+
// Realpath both sides so a symlink IN the repo, or a repo checked out
79+
// under a symlinked path (`/var` → `/private/var` on macOS), still
80+
// compares apples to apples. Missing realpath calls on the target fail
81+
// fast into the catch (no read attempted).
82+
const rootReal = await fs.realpath(root)
83+
const targetReal = await fs.realpath(path.join(root, rel))
84+
// Ensure `targetReal` is inside `rootReal` using a separator-aware
85+
// startsWith check so `/repo-backup` doesn't count as inside `/repo`.
86+
const sep = path.sep
87+
if (targetReal !== rootReal && !targetReal.startsWith(rootReal + sep)) return undefined
88+
return await fs.readFile(targetReal, "utf8")
89+
} catch {
90+
return undefined
91+
}
92+
}
93+
6594
/** Build a getContent(path, side) resolver over git refs / the working tree.
6695
* `renames` maps a new path → its old path so the "old" side of a renamed file
67-
* resolves from where it actually lived at `base` (not the post-rename path). */
68-
export function makeContentResolver(opts: CollectOptions & { renames?: Map<string, string> }) {
96+
* resolves from where it actually lived at `base` (not the post-rename path).
97+
* `gitRoot` is the repository top-level (from `git rev-parse --show-toplevel`).
98+
* When omitted, working-tree reads fall back to `opts.cwd`, which is only
99+
* correct when the CLI is invoked from the repo root. */
100+
export function makeContentResolver(opts: CollectOptions & { renames?: Map<string, string>; gitRoot?: string }) {
69101
return async (file: string, side: "old" | "new"): Promise<string | undefined> => {
70102
try {
71103
if (side === "old") {
@@ -75,13 +107,38 @@ export function makeContentResolver(opts: CollectOptions & { renames?: Map<strin
75107
if (opts.head) {
76108
return await git(["show", `${opts.head}:${file}`], opts.cwd)
77109
}
78-
return await fs.readFile(path.join(opts.cwd, file), "utf8")
110+
// File paths from `git diff --name-status` are repo-root relative,
111+
// NOT `opts.cwd` relative. When the CLI is invoked from a subdirectory
112+
// the naive `path.join(opts.cwd, file)` double-joins and fails ENOENT,
113+
// returning undefined and silently demoting downstream detectors to
114+
// the diff-only fallback. Root at the resolved git top-level when
115+
// supplied by the caller; fall back to `opts.cwd` when we couldn't
116+
// resolve it (non-git or bare-repo contexts). Reads are containment-
117+
// checked (symlink-safe) — see safeReadInside.
118+
const root = opts.gitRoot ?? opts.cwd
119+
return await safeReadInside(root, file)
79120
} catch {
80121
return undefined
81122
}
82123
}
83124
}
84125

126+
/** Resolve the repository top-level (`git rev-parse --show-toplevel`).
127+
* Used to root working-tree FS reads and existence checks at the repo root
128+
* regardless of the caller's cwd. Returns undefined outside a git repo.
129+
* Strips only the git-emitted terminator (`\r\n` or `\n`) rather than
130+
* `trim()` — a path with legitimate leading/trailing whitespace stays
131+
* intact (cubic-review P3). */
132+
export async function gitRepoRoot(cwd: string): Promise<string | undefined> {
133+
try {
134+
const out = await git(["rev-parse", "--show-toplevel"], cwd)
135+
const root = out.replace(/[\r\n]+$/, "")
136+
return root || undefined
137+
} catch {
138+
return undefined
139+
}
140+
}
141+
85142
/** Resolve a sensible default base ref (merge-base with origin/main/master). */
86143
export async function defaultBaseRef(cwd: string): Promise<string> {
87144
for (const candidate of ["origin/main", "origin/master", "main", "master"]) {

‎packages/opencode/src/altimate/review/orchestrate.ts‎

Lines changed: 58 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,10 @@ export interface OrchestrateInput {
190190
/** PR metadata passed to the AI reviewer for intent checking. */
191191
prTitle?: string
192192
prBody?: string
193+
/** G1 — attach the classifier's reason list to the envelope. */
194+
explainTier?: boolean
195+
/** G2 — override the classifier's tier decision. Envelope carries tierForced:true. */
196+
forceTier?: "trivial" | "lite" | "full"
193197
}
194198

195199
/** Derive the dbt model name from a model file path. */
@@ -1088,14 +1092,31 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
10881092
// A run is degraded when model files exist but none resolved against a manifest.
10891093
const runDegraded = modelFiles.length > 0 ? !anyManifest : reviewable.length === 0
10901094

1091-
const tier = classifyPR(reviewable, {
1095+
const tierResult = classifyPR(reviewable, {
10921096
blastRadiusOf: (p) => {
10931097
const c = ctxByPath.get(p)
10941098
return c ? c.impact.directCount + c.impact.transitiveCount : 0
10951099
},
10961100
touchesPiiOf: (f) => (ctxByPath.get(f.path)?.pii.length ?? 0) > 0,
10971101
isComplexOf: (f) => ctxByPath.get(f.path)?.complex ?? false,
1098-
}).tier
1102+
})
1103+
const classifiedTier = tierResult.tier
1104+
// G2 — --force-tier overrides the classifier. Envelope records both the
1105+
// forced tier and the original classification whenever the flag is passed
1106+
// (regardless of whether the forced value happens to match the classifier),
1107+
// so audits can see the bypass every time the flag was used. The reasons
1108+
// list gets a leading "forced" marker so downstream doesn't confuse the
1109+
// forced tier for a natural one. Codex-R18-review fix — the earlier
1110+
// `!== classifiedTier` gate silently hid the bypass when the forced tier
1111+
// matched the classifier's result.
1112+
const tier = input.forceTier ?? classifiedTier
1113+
const tierForced = input.forceTier !== undefined
1114+
const tierReasons = tierForced
1115+
? [
1116+
`forced via --force-tier=${input.forceTier} (classifier said ${classifiedTier})`,
1117+
...tierResult.reasons,
1118+
]
1119+
: tierResult.reasons
10991120

11001121
const lanes = new Set(input.config.reviewers.length ? input.config.reviewers : TIER_LANES[tier])
11011122

@@ -1196,8 +1217,38 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
11961217

11971218
// schema.yml-level detectors (test removal) — run on changed YAML files
11981219
// regardless of tier, since deleting a guardrail test is always worth flagging.
1199-
for (const file of reviewable) {
1200-
if (file.kind === "schema_yml") all.push(detectSchemaYmlPatterns(file, input.rubric))
1220+
// Pass old + new content when available so the detector diffs structural YAML
1221+
// (per-(entity, column, test) tuples) instead of walking the unified diff
1222+
// (which loses context when the model header is outside the -U3 window and
1223+
// silently drops sibling-column edge cases). Falls back to diff-only line
1224+
// detection when a content resolver isn't wired (e.g. unit-test callers).
1225+
//
1226+
// `oldContent` is fetched for anything that has an old side — MODIFIED,
1227+
// RENAMED, and DELETED. A schema.yml being renamed (e.g. moved to a new
1228+
// subdir) that also drops a `unique`/`not_null` guardrail must still surface
1229+
// as a finding; the earlier `status === "modified"` gate silently skipped
1230+
// renames. A whole schema.yml being DELETED removes every test declared in
1231+
// it — an even bigger removal — so the detector runs against `oldContent`
1232+
// vs an empty new document (cubic-review P2).
1233+
//
1234+
// `newContent` fetch is skipped for deleted files (nothing at HEAD to read;
1235+
// git-show would fail).
1236+
//
1237+
// Fetches run in parallel across schema files (a schema-heavy PR could touch
1238+
// dozens of yml files; serial `git show` per file adds up).
1239+
const schemaFiles = reviewable.filter((f) => f.kind === "schema_yml")
1240+
if (schemaFiles.length) {
1241+
const schemaFindingSets = await Promise.all(
1242+
schemaFiles.map(async (file) => {
1243+
const oldRef = file.oldPath ?? file.path
1244+
const [oldContent, newContent] = await Promise.all([
1245+
file.status !== "added" ? getContent?.(oldRef, "old") : Promise.resolve(undefined),
1246+
file.status !== "deleted" ? getContent?.(file.path, "new") : Promise.resolve(undefined),
1247+
])
1248+
return detectSchemaYmlPatterns(file, input.rubric, { oldContent, newContent })
1249+
}),
1250+
)
1251+
for (const findings of schemaFindingSets) all.push(findings)
12011252
}
12021253

12031254
// Architectural dedup: the regex `dbt-patterns`/`rule-catalog` layer is a
@@ -1349,6 +1400,9 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
13491400
manifestHash: input.manifestHash,
13501401
generatedAt: input.generatedAt,
13511402
degraded,
1403+
tierReasons: input.explainTier || tierForced ? tierReasons : undefined,
1404+
tierForced: tierForced ? true : undefined,
1405+
tierClassified: tierForced ? classifiedTier : undefined,
13521406
})
13531407
return signEnvelope(envelope)
13541408
}

0 commit comments

Comments
 (0)