Skip to content

Commit c06a851

Browse files
anandgupta42claude
andcommitted
test(drivers): assert the DuckDB store is a live engine, not just an open-looking connection
Every existing assertion in `duckdb-open-e2e.test.ts` can be satisfied by a connection that is not really there. A row count can legitimately be zero, and an absent error string can mean the error was swallowed rather than that none occurred — which is live on `main` today: `register.ts`'s `sql.execute` catches everything and returns `{ row_count: 0, error }` instead of throwing, and `formatResult` renders "(0 rows)" without ever reading `error`. So "(0 rows)" and "the driver is dead" are indistinguishable to anything downstream. `md5()` is not. A dead connection cannot return one row, and a live one cannot return the right digest without hashing the input, so this is a check whose correct answer cannot be produced without touching the thing under test. Added in two places: - The `beforeAll` gate, which already refused to run against a fake driver but did so on a row count a stub can trivially return. The digest is the half a stub cannot satisfy by returning a plausible shape, and it gates the whole file. - An explicit test after the store-open assertions, asserting both that exactly one row came back and that the digest is correct. The expected digest is computed independently (`hashlib.md5`, cross-checked against `md5(1)`), never by asking DuckDB — deriving it from the system under test would make the assertion circular. Not hypothetical for this PR: during the driver A/B, two binaries looked clean across transcripts, traces and 563 lines of debug output — zero ENOENT, zero fault lines — while failing to load the driver entirely. A probe of this shape is what caught it, after four earlier probe designs passed while proving nothing. **Verified the assertion is load-bearing rather than decorative**, since a liveness check that cannot fail is exactly the defect it exists to prevent: - DuckDB independently returns `517f58256b5ba4642643b3e884d91d15` for the nonce, matching the digest computed outside it. - A wrong digest fails the `beforeAll` gate and aborts the entire file (0 pass, 1 fail). - A wrong digest in the test alone, with the gate left intact, fails that test (13 pass, 1 fail). - Appending `WHERE false` to reproduce the "(0 rows)" shape that reads as success downstream also fails the test (13 pass, 1 fail) — the swallowed-error case specifically. Gates unchanged: typecheck 13/13; markers ok against `origin/main`; drivers 142 pass 0 fail; `test/altimate` 4224 pass 0 fail (one additional skip, the new gated test); gated E2E 22 pass 0 fail on six consecutive runs; lint 5864 warnings / 1 error, byte-identical to the previous commit — the error remains the pre-existing `consistent-return` in `packages/http-recorder/test/record-replay.test.ts`. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent 6ac0c64 commit c06a851

1 file changed

Lines changed: 41 additions & 2 deletions

File tree

‎packages/opencode/test/altimate/duckdb-open-e2e.test.ts‎

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,29 @@ let storePath = ""
3535
// creates `lockedPath` itself, so the only writer is the foreign process.
3636
let lockedPath = ""
3737

38+
// A liveness probe whose correct answer cannot be produced without a working
39+
// engine actually computing it.
40+
//
41+
// Every other assertion in this file can be satisfied by a connection that is
42+
// not really there. A row count can legitimately be zero, and an absent error
43+
// string can mean the error was swallowed rather than that none occurred — which
44+
// is live on `main` today: `register.ts`'s `sql.execute` catches everything and
45+
// returns `{ row_count: 0, error }` instead of throwing, and `formatResult`
46+
// renders "(0 rows)" without ever reading `error`. So "(0 rows)" and "the driver
47+
// is dead" are indistinguishable to anything downstream.
48+
//
49+
// `md5()` is not. A dead connection cannot return one row, and a live one cannot
50+
// return the right digest without hashing the input. This is not a hypothetical
51+
// safeguard: during the driver A/B for this PR, two binaries looked clean across
52+
// transcripts, traces and 563 lines of debug output — zero ENOENT, zero fault
53+
// lines — while failing to load the driver at all. This probe is what caught it.
54+
//
55+
// The digest is computed independently (`hashlib.md5`, cross-checked against
56+
// `md5(1)`), never by asking DuckDB — deriving it from the system under test
57+
// would make the assertion circular and prove nothing.
58+
const LIVENESS_NONCE = "altimate-duckdb-open-e2e-liveness-2026-08-30"
59+
const LIVENESS_MD5 = "517f58256b5ba4642643b3e884d91d15"
60+
3861
/** Run `fn` and return the error message it threw, or "" if it succeeded. */
3962
async function messageFrom(fn: () => Promise<unknown>): Promise<string> {
4063
try {
@@ -55,10 +78,12 @@ describe("DuckDB driver: opening a real store", () => {
5578
await c.connect()
5679
await c.execute("CREATE TABLE t AS SELECT 1 AS a, 'x' AS b")
5780
const probe = await c.execute("SELECT count(*) AS n FROM t")
81+
const liveness = await c.execute(`SELECT md5('${LIVENESS_NONCE}') AS h`)
5882
await c.close()
5983
// Refuse to run against anything but the real thing. A fake driver would
60-
// pass most assertions below while proving nothing.
61-
if (Number(probe.rows?.[0]?.[0]) !== 1) {
84+
// pass most assertions below while proving nothing. The digest is the half
85+
// of this gate that a stub cannot satisfy by returning a plausible shape.
86+
if (Number(probe.rows?.[0]?.[0]) !== 1 || String(liveness.rows?.[0]?.[0]) !== LIVENESS_MD5) {
6287
throw new Error(
6388
"ALTIMATE_DUCKDB_E2E=1 but the DuckDB driver is not the real one — " +
6489
"either the native binding is missing, or another test file has replaced " +
@@ -84,6 +109,20 @@ describe("DuckDB driver: opening a real store", () => {
84109
}
85110
})
86111

112+
ddbTest("the opened store is a live engine, not a connection that merely looks open", async () => {
113+
// The assertion above this one passes on a dead connection: `(0 rows)` and
114+
// "the driver never loaded" are the same observation. This one cannot.
115+
const c = await connect({ type: "duckdb", path: storePath })
116+
await c.connect()
117+
const r = await c.execute(`SELECT md5('${LIVENESS_NONCE}') AS h`)
118+
// Exactly one row: a swallowed failure surfaces as zero rows, and zero rows
119+
// is the shape that reads as success everywhere downstream.
120+
expect(r.rows.length).toBe(1)
121+
// And the right answer, which requires actually computing it.
122+
expect(String(r.rows[0][0])).toBe(LIVENESS_MD5)
123+
await c.close()
124+
})
125+
87126
ddbTest("opens the same store from many connectors at once", async () => {
88127
const results = await Promise.all(
89128
Array.from({ length: 8 }, async () => {

0 commit comments

Comments
 (0)