Skip to content

Commit 7f5ebb9

Browse files
anandgupta42claude
andcommitted
fix(drivers): address #1238 review — directory/create ordering, file: URI slashes, Windows paths, telemetry mislabel, post-probe failure
Follow-up to #1238's own review round (8 threads, all on the new code that PR added). Verified each against current code before acting; fixed the real bugs with tests, replied with reasoning on the one genuine design call. - file-store.ts: `create: true` bypassed the new directory check entirely, since `if (allowCreate) return` ran before it — a directory path with `create: true` reached the driver instead of getting this guard's clear error. Extracted `rejectIfDirectory()` and moved it before the `allowCreate` bypass in both the plain-path and `file:` URI branches, since neither engine can create a database at a path that's already a directory. - file-store.ts `absoluteFileUriPath`: the leading-slash check was bounded to 1-3 slashes, so `file:////mnt/share/warehouse.duckdb` (UNC-style, 4+ slashes) was treated as relative and bypassed the existence guard. Verified `fileURLToPath` handles any number of leading slashes without throwing; widened the check to `/^\/+/`. - file-store.ts `isLocalFilePath`: the `scheme://` regex accepted a single-character scheme, so a doubled-slash Windows path like `C://data/warehouse.duckdb` matched it exactly like `s3://...` and was misclassified as remote. No real scheme is a single letter; require 2+ characters before `://`. - sql-execute.ts: the previous round's "fingerprint on every outcome" fix mislabeled the catch block. `sql.execute` never throws for a connection/query failure — it returns a result carrying `error`, already fingerprinted on the result-error branch. The catch block only fires when the query never reached a warehouse at all (e.g. dispatcher down), so fingerprinting it there folded "never executed" into a signal meant to measure "executed SQL", re-biasing the telemetry in the opposite direction. Removed the catch-block emission; kept it on success + result-error only. - drivers-e2e.test.ts: if DuckDB setup failed after the availability probe already passed, every test's `if (!duckdbReady) return` guard reported a pass instead of a failure — the same vacuous-green class as the false-skip bug, one layer down. Extracted `connectWithRetry()`, which now throws (with the underlying error) once retries are exhausted instead of swallowing it; `beforeAll` no longer catches that throw, so a genuine post-probe failure now fails the whole describe block. Added a standalone unit test for the helper that runs independent of real DuckDB availability. Not fixed — genuine design call, replied with reasoning, left open: a custom DuckDB extension using an unlisted bare `scheme:` form (e.g. `acme:catalog`) is no longer forwarded, because narrowing the bare-scheme exclusion to a closed list (to fix the `data:warehouse.duckdb` false-positive from the prior round) is in direct, unavoidable tension with forwarding arbitrary unknown bare schemes — nothing in the syntax alone can tell them apart. Gates: `bun test packages/drivers/` 323/323 pass, `bun test packages/opencode/test/altimate/` 5009/5009 pass (653 skip, unrelated), `bun turbo typecheck` 13/13, marker check `--base origin/main --strict` clean, prettier clean on every newly-written/substantially-edited file. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent b96f987 commit 7f5ebb9

5 files changed

Lines changed: 231 additions & 56 deletions

File tree

‎packages/drivers/src/file-store.ts‎

Lines changed: 49 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,15 @@ const NON_SLASH_REMOTE_SCHEMES = ["md:", "motherduck:", "ducklake:"]
5757
export function isLocalFilePath(dbPath: string): boolean {
5858
if (dbPath === "" || dbPath === ":memory:") return false
5959
if (/^file:/i.test(dbPath)) return false
60-
if (/^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//.test(dbPath)) return false
60+
// altimate_change start — a single-letter "scheme" is a Windows drive letter, not a URI
61+
// A doubled-slash Windows path like `C://data/warehouse.duckdb` is a valid absolute
62+
// path (path.win32.normalize collapses it to `C:\data\warehouse.duckdb`), but the
63+
// scheme://-form regex below used to accept a one-character scheme, so `C:` matched
64+
// it exactly like `s3:` does. No real remote/extension scheme is a single letter —
65+
// require at least two characters before "://" so a drive letter is never mistaken
66+
// for one.
67+
if (/^[a-zA-Z][a-zA-Z0-9+.-]+:\/\//.test(dbPath)) return false
68+
// altimate_change end
6169
if (NON_SLASH_REMOTE_SCHEMES.some((scheme) => dbPath.toLowerCase().startsWith(scheme))) return false
6270
return true
6371
}
@@ -82,7 +90,13 @@ export function absoluteFileUriPath(dbPath: string): string | undefined {
8290
const rest = dbPath.slice("file:".length)
8391
if (rest === "" || rest.startsWith(":")) return undefined // file:, file::memory:
8492
if (/[?&]mode=memory\b/i.test(dbPath)) return undefined
85-
const isSlashForm = /^\/{1,3}/.test(rest)
93+
// altimate_change start — accept any number of leading slashes, not just 1-3.
94+
// `file:////mnt/share/warehouse.duckdb` (four or more slashes — seen with UNC-style
95+
// shares) is a valid absolute file: URI; fileURLToPath handles the extra slashes by
96+
// folding them into the resulting path (verified: `file:////x` -> `//x`), so there is
97+
// no reason to reject it here before even trying to parse it.
98+
const isSlashForm = /^\/+/.test(rest)
99+
// altimate_change end
86100
const isBareWindowsDrive = /^[a-zA-Z]:[\\/]/.test(rest)
87101
if (!isSlashForm && !isBareWindowsDrive) return undefined // relative — not this guard's job
88102
try {
@@ -126,29 +140,40 @@ export function allowsCreate(config: ConnectionConfig): boolean {
126140
return config.create === true
127141
}
128142

129-
// altimate_change start — a directory at dbPath is not a valid store either
143+
// altimate_change start — a directory at dbPath is never a valid store, with or
144+
// without `create`
130145
/**
131-
* Whether `path` names a store file that actually exists. `fs.existsSync`
132-
* alone is also true for a directory at that path, which the driver would
133-
* then fail to open with a confusing engine-level error instead of this
134-
* guard's clear one — a config that names a directory (a typo, or a path one
135-
* level off) should be reported the same way a missing file is.
146+
* Throw if `path` names an existing directory. A directory can never be
147+
* opened as a database by either engine, whether or not the caller passed
148+
* `create: true` — creation only ever means "create a missing FILE", not
149+
* "replace a directory". This must run before the `allowCreate` bypass in
150+
* `assertStoreExists`: `create: true` is meant to skip the "does this file
151+
* exist yet" check, not the "is this actually a directory" check, or a
152+
* misconfigured directory path reaches the driver and fails there with a
153+
* confusing engine-level error instead of this guard's clear one.
136154
*/
137-
function existsAsFile(path: string): boolean {
138-
if (!fs.existsSync(path)) return false
155+
function rejectIfDirectory(path: string, displayPath: string, engine: string): void {
156+
let isDir: boolean
139157
try {
140-
return !fs.statSync(path).isDirectory()
158+
isDir = fs.existsSync(path) && fs.statSync(path).isDirectory()
141159
} catch {
142-
// A stat failure (e.g. a race with a delete, or a permissions error)
160+
// A stat failure here (e.g. a race with a delete, or a permissions error)
143161
// is not this guard's to diagnose — let the driver's own open surface it.
144-
return true
162+
isDir = false
145163
}
164+
if (!isDir) return
165+
throw new Error(
166+
`${engine} database path is a directory, not a file: "${displayPath}". ` +
167+
`A directory can never be opened as a database — this applies even when "create" is set, ` +
168+
`since creation only ever means creating a missing file.`,
169+
)
146170
}
147171
// altimate_change end
148172

149173
/**
150-
* Throw unless the store is safe to open: it already exists, the caller opted
151-
* in to creating it, or the path is not a local file at all.
174+
* Throw unless the store is safe to open: it is not a directory, and it
175+
* either already exists, the caller opted in to creating it, or the path is
176+
* not a local file at all.
152177
*
153178
* @param engine Human-readable engine name used in the error message.
154179
* @param allowCreate Whether this open will actually create the store. Defaults
@@ -161,7 +186,6 @@ export function assertStoreExists(
161186
engine: string,
162187
allowCreate: boolean = allowsCreate(config),
163188
): void {
164-
if (allowCreate) return
165189
// altimate_change start — existence-check an absolute `file:` URI too;
166190
// `isLocalFilePath` deliberately excludes `file:` (see its own comment), so
167191
// without this branch every `file:` store — including absolute ones that
@@ -170,7 +194,11 @@ export function assertStoreExists(
170194
// the exact bug class this guard exists to catch.
171195
const fileUriPath = absoluteFileUriPath(dbPath)
172196
if (fileUriPath !== undefined) {
173-
if (existsAsFile(fileUriPath)) return
197+
// altimate_change: the directory check MUST run before the `allowCreate`
198+
// bypass below — see rejectIfDirectory's own comment.
199+
rejectIfDirectory(fileUriPath, dbPath, engine)
200+
if (allowCreate) return
201+
if (fs.existsSync(fileUriPath)) return
174202
throw new Error(
175203
`${engine} database file not found: "${dbPath}" (resolved to "${fileUriPath}"). ` +
176204
`Opening a warehouse connection never creates the database — an empty store would answer every query with no rows. ` +
@@ -179,7 +207,10 @@ export function assertStoreExists(
179207
}
180208
// altimate_change end
181209
if (!isLocalFilePath(dbPath)) return
182-
if (existsAsFile(dbPath)) return
210+
// altimate_change: same ordering requirement as the file: URI branch above.
211+
rejectIfDirectory(dbPath, dbPath, engine)
212+
if (allowCreate) return
213+
if (fs.existsSync(dbPath)) return
183214
throw new Error(
184215
`${engine} database file not found: "${dbPath}". ` +
185216
`Opening a warehouse connection never creates the database — an empty store would answer every query with no rows. ` +

‎packages/drivers/test/file-store-guard.test.ts‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,23 @@ describe("isLocalFilePath", () => {
4949
expect(isLocalFilePath("C:\\data\\warehouse.duckdb")).toBe(true)
5050
})
5151

52+
// altimate_change start — regression: a doubled-slash Windows drive path is a
53+
// real local path, not a `scheme://` URI
54+
test("treats a doubled-slash Windows drive path as local, not a scheme:// URI", () => {
55+
// `C://data/warehouse.duckdb` is a valid (if unusual) absolute Windows path —
56+
// path.win32.normalize collapses it to `C:\data\warehouse.duckdb`. The
57+
// scheme://-form regex used to accept a single-character scheme, so "C"
58+
// matched it exactly like "s3" does in `s3://...`, misclassifying it as
59+
// remote and skipping both path resolution and the existence guard. No real
60+
// remote/extension scheme is a single letter, so requiring 2+ characters
61+
// before "://" fixes this without affecting genuine schemes.
62+
expect(isLocalFilePath("C://data/warehouse.duckdb")).toBe(true)
63+
expect(isLocalFilePath("D://warehouse.duckdb")).toBe(true)
64+
// Genuine two-or-more-character schemes are still excluded.
65+
expect(isLocalFilePath("s3://bucket/warehouse.duckdb")).toBe(false)
66+
})
67+
// altimate_change end
68+
5269
test("only the exact `:memory:` is in-memory — `:memory:name` is a real file", () => {
5370
// DuckDB writes a file literally named ":memory:named" for this path, and
5471
// a colon-prefixed name is an ordinary file to both engines. Classifying
@@ -124,6 +141,16 @@ describe("assertStoreExists", () => {
124141
expect(() => assertStoreExists({ type: "duckdb" }, uri, "DuckDB")).not.toThrow()
125142
})
126143

144+
// altimate_change: regression — a `file:////...` URI (4+ leading slashes)
145+
// used to be misread as relative by the bounded {1,3}-slash check, so
146+
// isLocalFilePath's own file: exclusion made assertStoreExists skip it
147+
// entirely and a missing store there opened silently empty.
148+
test("throws for a missing `file:////` (4-slash) absolute URI", () => {
149+
const missing = path.join(tmp(), "absent.duckdb")
150+
const uri = `file:///${missing}` // absoluteFileUriPath sees 4 total slashes after "file:"
151+
expect(() => assertStoreExists({ type: "duckdb" }, uri, "DuckDB")).toThrow("not found")
152+
})
153+
127154
test("does not existence-check a relative `file:` URI (tracked separately as #1209)", () => {
128155
// resolveStorePaths leaves relative file: URIs untouched, so guarding
129156
// existence here would check against whatever the process cwd happens to
@@ -137,7 +164,37 @@ describe("assertStoreExists", () => {
137164
// altimate_change start — a directory at dbPath is not a valid store
138165
test("throws when dbPath names a directory, not a file", () => {
139166
const dir = tmp()
140-
expect(() => assertStoreExists({ type: "duckdb" }, dir, "DuckDB")).toThrow("not found")
167+
// altimate_change: the message now specifically says "directory" (a more
168+
// accurate diagnosis than "not found" — the path DOES exist), since
169+
// rejectIfDirectory reports it before the exists/missing check ever runs.
170+
expect(() => assertStoreExists({ type: "duckdb" }, dir, "DuckDB")).toThrow("directory")
171+
})
172+
173+
// altimate_change: regression — `create: true` used to bypass the directory
174+
// check entirely (the `if (allowCreate) return` ran before it), so a
175+
// directory path reached the driver with `create: true` and failed there
176+
// with a confusing engine-level error instead of this guard's clear one.
177+
// Neither engine can create a database AT a path that is already a
178+
// directory, so the directory rejection must fire regardless of `create`.
179+
test("throws when dbPath names a directory even with create: true", () => {
180+
const dir = tmp()
181+
expect(() => assertStoreExists({ type: "duckdb", create: true }, dir, "DuckDB")).toThrow("directory")
182+
// A plain "not found" would be misleading here — the path DOES exist, it's
183+
// just not a valid store — so the message must say "directory", not "not found".
184+
expect(() => assertStoreExists({ type: "duckdb", create: true }, dir, "DuckDB")).not.toThrow("not found")
185+
})
186+
187+
test("still allows creation of a missing FILE with create: true (directory check doesn't over-reject)", () => {
188+
const dir = tmp()
189+
expect(() =>
190+
assertStoreExists({ type: "duckdb", create: true }, path.join(dir, "new.duckdb"), "DuckDB"),
191+
).not.toThrow()
192+
})
193+
194+
test("throws when an absolute `file:` URI names a directory even with create: true", () => {
195+
const dir = tmp()
196+
const uri = `file://${dir}`
197+
expect(() => assertStoreExists({ type: "duckdb", create: true }, uri, "DuckDB")).toThrow("directory")
141198
})
142199
// altimate_change end
143200
})
@@ -151,6 +208,16 @@ describe("absoluteFileUriPath", () => {
151208
expect(absoluteFileUriPath("file:relative/warehouse.duckdb")).toBeUndefined()
152209
})
153210

211+
// altimate_change start — regression: 4+ leading slashes (UNC-style shares)
212+
// used to be rejected as "not absolute" by a bounded {1,3} slash count; any
213+
// number of leading slashes is a valid absolute file: URI form, and
214+
// fileURLToPath folds the extra slashes into the resulting path.
215+
test("resolves an absolute `file:` URI with four or more leading slashes", () => {
216+
expect(absoluteFileUriPath("file:////mnt/share/warehouse.duckdb")).toBe("//mnt/share/warehouse.duckdb")
217+
expect(absoluteFileUriPath("file://///mnt/share/warehouse.duckdb")).toBe("///mnt/share/warehouse.duckdb")
218+
})
219+
// altimate_change end
220+
154221
test("returns undefined for the in-memory/temporary forms", () => {
155222
expect(absoluteFileUriPath("file:")).toBeUndefined()
156223
expect(absoluteFileUriPath("file::memory:")).toBeUndefined()

‎packages/opencode/src/altimate/tools/sql-execute.ts‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -122,10 +122,14 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
122122
})
123123
} catch (e) {
124124
const msg = e instanceof Error ? e.message : String(e)
125-
// altimate_change start — fingerprint a thrown-exception failure too (see the
126-
// matching comment on the result-error branch above)
127-
emitSqlFingerprint(args.query, ctx.sessionID)
128-
// altimate_change end
125+
// altimate_change: deliberately NOT fingerprinted. This catch only fires when
126+
// `Dispatcher.call` itself throws (dispatcher down, no warehouse configured) —
127+
// per the comment on the result-error branch above, `sql.execute` never throws
128+
// for a connection/query failure, it returns a result carrying `error`, which
129+
// that branch already fingerprints. A query that reaches here never ran against
130+
// any warehouse, so fingerprinting it here would fold "never executed" into a
131+
// signal meant to measure "executed SQL" (success and result-error), re-biasing
132+
// the telemetry this change is meant to correct in the opposite direction.
129133
// altimate_change — annotate the failure too. A fail-open notice that only rides
130134
// on success is worse than none: the reason vanishes exactly when the call went
131135
// wrong, and the `precedence` marker under-counts fail-open in precisely the
@@ -139,9 +143,15 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
139143
},
140144
})
141145

142-
// altimate_change start — emit SQL structure fingerprint telemetry on every
143-
// execution outcome (success, a result-shaped error, or a thrown exception),
144-
// not only on success. Extracted so all three call sites stay in sync.
146+
// altimate_change start — emit SQL structure fingerprint telemetry for every
147+
// outcome where a warehouse actually ran the query: success, and a result-shaped
148+
// error (sql.execute returns `{ ..., error }` rather than throwing for a
149+
// connection/query failure — see the result-error branch above). Deliberately
150+
// NOT called from the thrown-exception catch block: that path only fires when
151+
// the query never reached a warehouse at all (e.g. dispatcher down), and
152+
// fingerprinting a never-executed query there would bias this "executed SQL
153+
// structure" signal toward attempts that never ran. Extracted so the two
154+
// legitimate call sites stay in sync.
145155
function emitSqlFingerprint(query: string, sessionID: string): void {
146156
try {
147157
const fp = computeSqlFingerprint(query)

‎packages/opencode/test/altimate/drivers-e2e.test.ts‎

Lines changed: 78 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,65 @@ async function waitForPort(
6060
throw new Error(`Port ${port} not reachable after ${timeoutMs}ms`)
6161
}
6262

63+
// altimate_change start — retry a flaky setup step, and fail loudly (not silently)
64+
// once retries are exhausted.
65+
/**
66+
* Run `attempt` up to `maxAttempts` times, with a short backoff between tries,
67+
* and return its result on the first success. If every attempt fails, THROW
68+
* the last error rather than swallowing it.
69+
*
70+
* This is the difference between a genuine "not available here" (handled
71+
* elsewhere, before this is ever called) and "was available but setup broke":
72+
* a caller that catches this and silently leaves some "ready" flag false
73+
* recreates the exact vacuous-green failure this file's `probeDuckDB` exists
74+
* to prevent, one layer down — every test gated on that flag would then
75+
* report as passing via an early `if (!ready) return` instead of failing.
76+
*/
77+
async function connectWithRetry<T>(attempt: (attemptNumber: number) => Promise<T>, maxAttempts: number): Promise<T> {
78+
let lastError: unknown
79+
for (let n = 1; n <= maxAttempts; n++) {
80+
try {
81+
return await attempt(n)
82+
} catch (e) {
83+
lastError = e
84+
if (n < maxAttempts) await new Promise((r) => setTimeout(r, 100 * n))
85+
}
86+
}
87+
throw new Error(
88+
`Setup failed after ${maxAttempts} attempts: ${lastError instanceof Error ? lastError.message : String(lastError)}`,
89+
)
90+
}
91+
// altimate_change end
92+
93+
// altimate_change start — unit-test the retry-then-fail-loudly behavior directly,
94+
// independent of real DuckDB availability, so this regression is caught even in
95+
// environments where the DuckDB binding isn't installed at all.
96+
describe("connectWithRetry", () => {
97+
test("throws (does not silently resolve) once every attempt is exhausted", async () => {
98+
let calls = 0
99+
const alwaysFails = async () => {
100+
calls++
101+
throw new Error("transient setup failure")
102+
}
103+
await expect(connectWithRetry(alwaysFails, 3)).rejects.toThrow("Setup failed after 3 attempts")
104+
await expect(connectWithRetry(alwaysFails, 3)).rejects.toThrow("transient setup failure")
105+
expect(calls).toBe(6) // 3 attempts per call above, called twice
106+
})
107+
108+
test("resolves with the first successful attempt's result, retrying past earlier failures", async () => {
109+
let calls = 0
110+
const succeedsOnThirdTry = async () => {
111+
calls++
112+
if (calls < 3) throw new Error("not yet")
113+
return "connected"
114+
}
115+
const result = await connectWithRetry(succeedsOnThirdTry, 5)
116+
expect(result).toBe("connected")
117+
expect(calls).toBe(3)
118+
})
119+
})
120+
// altimate_change end
121+
63122
// altimate_change start — authoritative DuckDB availability probe.
64123
// `require("duckdb")` (isDuckDBAvailable) can return true when the native binding
65124
// is present in the process module cache but actually fails to CONNECT in this
@@ -113,31 +172,27 @@ describe("DuckDB Driver E2E", () => {
113172
let duckdbReady = false
114173

115174
// altimate_change start — retry DuckDB connection initialization to handle
116-
// transient native binding load failures when the full suite runs in parallel
175+
// transient native binding load failures when the full suite runs in parallel,
176+
// but FAIL (don't silently skip) if it never recovers.
177+
//
178+
// `probeDuckDB()` above already proved DuckDB is genuinely available and
179+
// working in this process. If setup here still fails after retries, that is
180+
// a real regression, not "DuckDB isn't available" — every test below still
181+
// runs (test.skipIf keys off `duckdbAvailable`, which stays true regardless
182+
// of what happens here), and each one used to just `if (!duckdbReady) return`
183+
// and report as passing: the same vacuous-green class the driver-e2e
184+
// false-skip fix removed, one layer down. Throwing here fails the whole
185+
// describe block instead of letting every test silently "pass" via that
186+
// early return.
117187
beforeAll(async () => {
118188
if (!duckdbAvailable) return
119-
const maxAttempts = 3
120-
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
121-
try {
122-
const mod = await import("@altimateai/drivers/duckdb")
123-
// altimate_change start — requireStorePath() now rejects a missing path
124-
connector = await mod.connect({ type: "duckdb", path: ":memory:" })
125-
// altimate_change end
126-
await connector.connect()
127-
duckdbReady = true
128-
break
129-
} catch (e) {
130-
if (attempt < maxAttempts) {
131-
// Brief delay before retry to let concurrent native-binding loads settle
132-
await new Promise((r) => setTimeout(r, 100 * attempt))
133-
} else {
134-
console.warn(
135-
"DuckDB not available (native binding may be missing); skipping DuckDB tests:",
136-
(e as Error).message,
137-
)
138-
}
139-
}
140-
}
189+
connector = await connectWithRetry(async () => {
190+
const mod = await import("@altimateai/drivers/duckdb")
191+
const c = await mod.connect({ type: "duckdb", path: ":memory:" })
192+
await c.connect()
193+
return c
194+
}, 3)
195+
duckdbReady = true
141196
})
142197
// altimate_change end
143198

0 commit comments

Comments
 (0)