Skip to content

Commit e700a40

Browse files
committed
test(review): PR #1041 adversarial coverage for fixes + repair vacuous tests
Two new positive tests pin the fixes from the previous commit: - `reviewPullRequest defaults cliVersion so the dbt_pr_review tool path gets audit provenance too` — invokes `reviewPullRequest` WITHOUT passing `cliVersion`, asserts `env.engine.cliVersion` is non-empty. - `deleted manifest-affecting file trips staleness even though it can't be stat'd` — deletion of `models/orders.sql` with the manifest present must set `env.staleManifest === true`. Two vacuous tests from the original v0.9.3 adversarial file are repaired per cubic-review PR #1041: - Grandparent walk-past regression test now plants an actual `grand/ target/manifest.json` (a poisoned grandparent DAG), then invokes review from `grand/inner/subdir` where `inner/dbt_project.yml` has no compiled artifact. The assertion `env.manifestHash` must remain undefined AND the stderr breadcrumb must NOT reference the grandparent path — a regressed walker would announce a hit there. Previously the fixture just made an unrelated sibling directory, so the walker never had a real manifest to consider skipping. - Symlink-loop test is removed. Its `looped -> tmp` fixture was a one-hop parent alias, not a cycle; `fs.realpath` resolves it in one step, so the test proved nothing about the try/catch fallback in `run.ts`. A real ELOOP fixture is filesystem-dependent and the fallback pattern is a straight try/catch obvious from a code read; cheaper to delete than to churn on a cross-platform cycle.
1 parent a3e2a89 commit e700a40

1 file changed

Lines changed: 106 additions & 49 deletions

File tree

‎packages/opencode/test/skill/release-v0.9.3-adversarial.test.ts‎

Lines changed: 106 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
*/
2323

2424
import { describe, test, expect } from "bun:test"
25-
import { mkdtemp, mkdir, writeFile, symlink, utimes } from "node:fs/promises"
25+
import { mkdtemp, mkdir, writeFile, utimes } from "node:fs/promises"
2626
import os from "node:os"
2727
import path from "node:path"
2828
import {
@@ -83,6 +83,33 @@ describe("v0.9.3 release: envelope audit fields (Compliance P0/P1)", () => {
8383
expect(verifyEnvelope(tampered, "k")).toBe(false)
8484
})
8585

86+
test("reviewPullRequest defaults cliVersion so the dbt_pr_review tool path gets audit provenance too", async () => {
87+
// Regression pin for cubic-review PR #1041: only the CLI command explicitly
88+
// forwarded Installation.VERSION, and the dbt_pr_review tool wrapper
89+
// never did — so agent-invoked verdicts silently dropped engine.cliVersion
90+
// and defeated the audit-trail promise of the field. The fix defaults the
91+
// field to Installation.VERSION inside reviewPullRequest so BOTH entry
92+
// points get it. This test invokes reviewPullRequest WITHOUT passing
93+
// cliVersion and asserts the field is populated on the returned envelope.
94+
const tmp = await mkdtemp(path.join(os.tmpdir(), "v093-clivers-"))
95+
const origWrite = process.stderr.write.bind(process.stderr)
96+
process.stderr.write = (() => true) as typeof process.stderr.write
97+
try {
98+
const env = await reviewPullRequest({
99+
cwd: tmp,
100+
head: undefined,
101+
// opts.cliVersion INTENTIONALLY OMITTED — simulates the tool-path caller.
102+
changedFiles: [{ path: "models/x.sql", status: "modified", diff: "+select 1\n" }],
103+
getContent: async () => "select 1",
104+
})
105+
expect(env.engine.cliVersion).toBeDefined()
106+
expect(typeof env.engine.cliVersion).toBe("string")
107+
expect(env.engine.cliVersion!.length).toBeGreaterThan(0)
108+
} finally {
109+
process.stderr.write = origWrite
110+
}
111+
})
112+
86113
test("staleManifest true is signed; verify fails if flipped after signing", () => {
87114
const signed = signEnvelope(
88115
buildEnvelope({
@@ -240,6 +267,36 @@ describe("v0.9.3 release: stale-manifest signal on envelope (DE/Compliance P1)",
240267
}
241268
})
242269

270+
test("deleted manifest-affecting file trips staleness even though it can't be stat'd (cubic PR #1041)", async () => {
271+
// Bug caught by cubic on PR #1041: the pre-fix detectStaleManifest took
272+
// `changedPaths: string[]` and tried to `stat(deletedFile)` which throws
273+
// silently, so a deleted `models/orders.sql` reported CLEAN even though
274+
// the manifest still references it — a false-negative that would
275+
// signed-envelope-certify a verdict against ghost models. This test
276+
// pins the fixed semantic: a deletion of a manifest-affecting path
277+
// must fire the stale signal AND land on the envelope.
278+
const tmp = await mkdtemp(path.join(os.tmpdir(), "v093-stale-del-"))
279+
await mkdir(path.join(tmp, "target"), { recursive: true })
280+
const manifestPath = path.join(tmp, "target", "manifest.json")
281+
await writeFile(manifestPath, JSON.stringify({ metadata: { adapter_type: "duckdb" } }))
282+
// NOTE: no `models/orders.sql` on disk — it was DELETED by this diff.
283+
const origWrite = process.stderr.write.bind(process.stderr)
284+
process.stderr.write = (() => true) as typeof process.stderr.write
285+
try {
286+
const env = await reviewPullRequest({
287+
cwd: tmp,
288+
head: undefined,
289+
manifestPath,
290+
changedFiles: [{ path: "models/orders.sql", status: "deleted", diff: "-select 1\n" }],
291+
getContent: async () => "",
292+
cliVersion: "0.9.3",
293+
})
294+
expect(env.staleManifest).toBe(true)
295+
} finally {
296+
process.stderr.write = origWrite
297+
}
298+
})
299+
243300
test("touching a non-manifest-affecting file (README.md) must NOT trip the stale signal", async () => {
244301
// Guard on `isManifestAffecting` — the mtime check MUST skip repo-wide
245302
// paths (README, package.json, .github/) or every unrelated commit
@@ -295,73 +352,73 @@ describe("v0.9.3 release: manifest auto-discovery hostile-shape resilience", ()
295352
}
296353
})
297354

298-
test("dbt_project.yml present but no target/manifest.json → no walk-past, returns undefined manifest", async () => {
355+
test("dbt_project.yml present but no target/manifest.json → no walk-past to grandparent's compiled DAG", async () => {
299356
// The auto-discovery walk was hardened in PR #1027 to NOT climb into a
300357
// grandparent's `target/` when the current dbt project just hasn't been
301358
// compiled. Reviewing against the wrong project's DAG would be worse
302359
// than lint-only. Guard against a regression that re-introduces the
303360
// silent fallback.
304-
const tmp = await mkdtemp(path.join(os.tmpdir(), "v093-uncompiled-"))
305-
await writeFile(path.join(tmp, "dbt_project.yml"), "name: test\nversion: 1.0.0\n")
306-
// Also fake a grandparent with a `target/manifest.json` — the walk must
307-
// NOT reach for it.
308-
const grandTarget = path.join(tmp, "..", "grandparent-target-fake")
309-
// Skip the grandparent poison if the tmpdir parent isn't writable
310-
// (some CI sandboxes) — the primary assertion still holds.
311-
try {
312-
await mkdir(grandTarget, { recursive: true })
313-
} catch {
314-
/* ignore */
315-
}
316-
const nested = path.join(tmp, "subdir")
317-
await mkdir(nested, { recursive: true })
361+
//
362+
// Fixture layout to actually exercise the walk-past guard (cubic-review
363+
// PR #1041 — the prior test only planted an unrelated sibling directory,
364+
// never a real grandparent manifest, so the walker never had one to
365+
// even consider skipping):
366+
//
367+
// grand/
368+
// dbt_project.yml ← poisoned grandparent project
369+
// target/manifest.json ← poisoned grandparent manifest
370+
// inner/
371+
// dbt_project.yml ← the project the reviewer should stop at
372+
// (no target/manifest.json — the guarded case)
373+
// subdir/ ← cwd
374+
const grand = await mkdtemp(path.join(os.tmpdir(), "v093-grand-"))
375+
await writeFile(path.join(grand, "dbt_project.yml"), "name: grand\nversion: 1.0.0\n")
376+
await mkdir(path.join(grand, "target"), { recursive: true })
377+
await writeFile(
378+
path.join(grand, "target", "manifest.json"),
379+
JSON.stringify({ metadata: { adapter_type: "duckdb" }, nodes: {} }),
380+
)
381+
const inner = path.join(grand, "inner")
382+
await mkdir(inner, { recursive: true })
383+
await writeFile(path.join(inner, "dbt_project.yml"), "name: inner\nversion: 1.0.0\n")
384+
const cwd = path.join(inner, "subdir")
385+
await mkdir(cwd, { recursive: true })
386+
318387
const origWrite = process.stderr.write.bind(process.stderr)
319-
process.stderr.write = (() => true) as typeof process.stderr.write
388+
let stderrBuf = ""
389+
process.stderr.write = ((s: string) => {
390+
stderrBuf += String(s)
391+
return true
392+
}) as typeof process.stderr.write
320393
try {
321394
const env = await reviewPullRequest({
322-
cwd: nested,
395+
cwd,
323396
head: undefined,
324-
// No --manifest → auto-discovery from subdir.
397+
// No --manifest → auto-discovery from cwd.
325398
changedFiles: [{ path: "models/x.sql", status: "modified", diff: "+select 1\n" }],
326399
getContent: async () => "select 1",
327400
cliVersion: "0.9.3",
328401
})
329402
expect(env.verdict).toBeDefined()
330-
// Silent fallback to a grandparent's target/ would leave manifestHash
403+
// The auto-discovery must have stopped at `inner`'s dbt_project.yml
404+
// and refused to reach for `grand/target/manifest.json`. Silent
405+
// fallback to the grandparent's target/ would leave manifestHash
331406
// populated with someone else's DAG.
332407
expect(env.manifestHash).toBeUndefined()
408+
// And the stderr "auto-discovered dbt manifest at ..." breadcrumb
409+
// must NOT reference the grandparent path — a regressed walker
410+
// would announce a hit there.
411+
expect(stderrBuf).not.toContain(path.join(grand, "target", "manifest.json"))
333412
} finally {
334413
process.stderr.write = origWrite
335414
}
336415
})
337416

338-
test("symlink loop in cwd path resolves via realpath and does not hang", async () => {
339-
// Guard against a regression in the realpath handling around the
340-
// dbtRoot / gitRoot canonicalization — a symlink loop would either
341-
// throw or (in a naive walker) infinite-loop. reviewPullRequest must
342-
// reach a defined verdict either way.
343-
const tmp = await mkdtemp(path.join(os.tmpdir(), "v093-symlink-"))
344-
// Point `looped` at itself's parent — realpath call must not spin.
345-
const looped = path.join(tmp, "looped")
346-
try {
347-
await symlink(tmp, looped)
348-
} catch {
349-
// Some filesystems reject same-dir loops; skip if so.
350-
return
351-
}
352-
const origWrite = process.stderr.write.bind(process.stderr)
353-
process.stderr.write = (() => true) as typeof process.stderr.write
354-
try {
355-
const env = await reviewPullRequest({
356-
cwd: looped,
357-
head: undefined,
358-
changedFiles: [{ path: "models/x.sql", status: "modified", diff: "+select 1\n" }],
359-
getContent: async () => "select 1",
360-
cliVersion: "0.9.3",
361-
})
362-
expect(env.verdict).toBeDefined()
363-
} finally {
364-
process.stderr.write = origWrite
365-
}
366-
})
417+
// NOTE: the "symlink loop" test that lived here previously created a
418+
// single-hop parent alias (`looped -> tmp`) which `fs.realpath` resolves
419+
// in one step — not a cycle. It therefore proved nothing about the
420+
// realpath try/catch fallback in run.ts and was removed rather than
421+
// rewritten (a real ELOOP fixture is filesystem-dependent, and the
422+
// realpath fallback pattern is a straight try/catch obvious from a code
423+
// read). Removed on cubic-review PR #1041 feedback.
367424
})

0 commit comments

Comments
 (0)