Skip to content

Commit b96f987

Browse files
anandgupta42claude
andcommitted
fix(drivers): recover #1204 review debt — driver-e2e false-skip, file: URI checks, Windows paths, telemetry, docs
#1204 (fix/warehouse-store-path-resolution, merged 1caa234) shipped the core path-resolution fix but merged with only 2 of 47 review threads addressed. This recovers the real bugs and finishes the incomplete P1 left behind. Recovered from an orphaned agent worktree, re-applied and re-verified here: - drivers-e2e.test.ts: three bare connect({ type: "duckdb" }) calls threw under #1204's new requireStorePath() guard, so probeDuckDB() always self-caught and 39 of 50 tests in the file silently skipped even with the duckdb binding installed. Fixed by passing path: ":memory:". Verified independently: 11 pass/39 skip -> 36 pass/14 skip. - file-store.ts isLocalFilePath: the scheme-exclusion regex matched any "2+ letter prefix + colon", misreading a local filename shaped like data:warehouse.duckdb as a remote scheme. Narrowed to real scheme:// URIs plus DuckDB's specific non-slash extension schemes (md:, motherduck:, ducklake:). Added a regression unit test. Completed the incomplete P1 (absoluteFileUriPath() was written but never wired into assertStoreExists, so behavior was unchanged on main): - Wired absoluteFileUriPath() into assertStoreExists so a missing *absolute* file: URI now fails loudly instead of opening silently empty — the exact bug class #1204 exists to fix. Relative file: URIs are deliberately left alone (tracked separately as #1209). Added tests for both. Remaining live review threads, fixed with tests where testable: - registry.ts resolveStorePaths: POSIX path.isAbsolute() doesn't recognize C:\..., so a shared/migrated Windows config path got mangled by path.resolve. Fixed with path.isAbsolute() || path.win32.isAbsolute(). By-inspection only — no Windows CI. - warehouse-add.ts: tool description said a relative path resolves "against the directory of the config that declares it"; Registry.add() actually resolves against projectRoot() at add-time regardless of where it's persisted (the global config). Description corrected to match. - sql-execute.ts: SQL fingerprint telemetry was emitted only on the success path, so failed executions (including the new error-surfacing branch) never got fingerprinted — biasing telemetry away from failures. Extracted a shared emitSqlFingerprint() helper and call it from all three outcomes (success, result-shaped error, thrown exception). - docs/configure/warehouses.md: still said path was optional/omittable for DuckDB and SQLite in-memory; requireStorePath() now rejects that. Corrected in both the DuckDB and SQLite sections. - file-store.ts assertStoreExists: fs.existsSync is also true for a directory at that path, which would have let a misconfigured directory path through to a confusing driver-level error instead of this guard's clear one. Added an existsAsFile check. Out of scope: the registry cross-tenant-scoping P1 (registry.ts:80/:175, the process-global loaded state) is tracked separately as #1237 and needs its own design PR — not attempted here. Gates: bun test packages/drivers/ packages/opencode/test/altimate/ green (5007+317 pass, 0 fail), bun turbo typecheck 13/13, marker check --base origin/main --strict clean (no upstream-shared files touched), prettier clean on all newly-written content. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent 1caa234 commit b96f987

8 files changed

Lines changed: 247 additions & 42 deletions

File tree

‎docs/docs/configure/warehouses.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -261,7 +261,7 @@ If you're already authenticated via `gcloud`, omit `credentials_path`:
261261

262262
| Field | Required | Description |
263263
|-------|----------|-------------|
264-
| `path` | No | Database file path. Omit or use `":memory:"` for in-memory |
264+
| `path` | Yes | Database file path, or `":memory:"` for in-memory. Cannot be omitted — a missing `path` is rejected rather than silently falling back to `":memory:"` |
265265
| `create` | No | Create the database file if it is missing (default: `false`) |
266266

267267
!!! warning "The store must already exist"
@@ -461,7 +461,7 @@ If you're already authenticated via `gcloud`, omit `credentials_path`:
461461

462462
| Field | Required | Description |
463463
|-------|----------|-------------|
464-
| `path` | No | Database file path. Omit or use `":memory:"` for in-memory |
464+
| `path` | Yes | Database file path, or `":memory:"` for in-memory. Cannot be omitted — a missing `path` is rejected rather than silently falling back to `":memory:"` |
465465
| `readonly` | No | Open in read-only mode (default: `false`) |
466466
| `create` | No | Create the database file if it is missing (default: `false`) |
467467

‎packages/drivers/src/file-store.ts‎

Lines changed: 98 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,18 @@
1313
*/
1414

1515
import * as fs from "fs"
16+
import { fileURLToPath } from "url"
1617
import type { ConnectionConfig } from "./types"
1718

19+
// altimate_change start — narrow the scheme exclusion to genuine remote/extension targets
20+
/**
21+
* DuckDB extension schemes that take a bare `scheme:rest` form with no `//`
22+
* — MotherDuck (`md:`) and DuckLake (`ducklake:`) — and so cannot be told
23+
* apart from a local filename by the `://` check below.
24+
*/
25+
const NON_SLASH_REMOTE_SCHEMES = ["md:", "motherduck:", "ducklake:"]
26+
// altimate_change end
27+
1828
/**
1929
* Whether `dbPath` names a file on the local filesystem, and so can be
2030
* existence-checked before the driver opens it.
@@ -29,16 +39,63 @@ import type { ConnectionConfig } from "./types"
2939
* A scheme-qualified target is not a local file: MotherDuck (`md:`), object
3040
* storage (`s3://`), DuckLake, and any other scheme a DuckDB extension
3141
* provides. Those are left to the driver, which reports an unknown scheme as a
32-
* missing-extension error rather than silently creating anything. The pattern
33-
* requires two or more characters before the colon so a Windows drive letter
34-
* (`C:\data\wh.duckdb`) stays a path.
42+
* missing-extension error rather than silently creating anything.
43+
*
44+
* altimate_change: the exclusion used to fire on ANY two-or-more-letter
45+
* prefix followed by a colon, which misclassified an ordinary local filename
46+
* that happens to contain one — `data:warehouse.duckdb`, `foo:warehouse.db`
47+
* — as a remote target, silently skipping both path resolution and the
48+
* existence guard below. Only a `scheme://` URI or one of the specific
49+
* non-slash extension schemes DuckDB actually recognizes is excluded now; a
50+
* `C:\...` Windows drive letter still passes through unaffected, since
51+
* neither pattern matches it. `file:` is deliberately still excluded here —
52+
* it is a real local path, but resolving/existence-checking it is handled
53+
* separately (see `absoluteFileUriPath` below) because it is not safe to
54+
* treat as an ordinary path string (see registry.ts's `resolveStorePaths`,
55+
* which would otherwise mangle it with `path.resolve`).
3556
*/
3657
export function isLocalFilePath(dbPath: string): boolean {
3758
if (dbPath === "" || dbPath === ":memory:") return false
38-
if (/^[a-zA-Z][a-zA-Z0-9+.-]+:/.test(dbPath)) return false
59+
if (/^file:/i.test(dbPath)) return false
60+
if (/^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//.test(dbPath)) return false
61+
if (NON_SLASH_REMOTE_SCHEMES.some((scheme) => dbPath.toLowerCase().startsWith(scheme))) return false
3962
return true
4063
}
4164

65+
// altimate_change start — existence-check absolute `file:` URIs too
66+
/**
67+
* The on-disk path an ABSOLUTE `file:` URI names, or `undefined` if `dbPath`
68+
* is not a `file:` URI, is a relative one, or is one of SQLite/DuckDB's
69+
* in-memory or temporary URI forms (`file:`, `file::memory:`,
70+
* `file:name?mode=memory`) that never touch disk.
71+
*
72+
* Scoped deliberately narrow to the absolute case. This PR does not resolve a
73+
* relative `file:` URI against a base directory (registry.ts's
74+
* `resolveStorePaths` leaves `file:` paths untouched — see `isLocalFilePath`
75+
* above), so guarding a relative one's existence here would check whatever
76+
* the process's current directory happens to be, which is exactly the
77+
* cwd-following bug this PR exists to remove. An absolute `file:` URI names
78+
* one unambiguous location regardless of cwd, so it is safe to check.
79+
*/
80+
export function absoluteFileUriPath(dbPath: string): string | undefined {
81+
if (!/^file:/i.test(dbPath)) return undefined
82+
const rest = dbPath.slice("file:".length)
83+
if (rest === "" || rest.startsWith(":")) return undefined // file:, file::memory:
84+
if (/[?&]mode=memory\b/i.test(dbPath)) return undefined
85+
const isSlashForm = /^\/{1,3}/.test(rest)
86+
const isBareWindowsDrive = /^[a-zA-Z]:[\\/]/.test(rest)
87+
if (!isSlashForm && !isBareWindowsDrive) return undefined // relative — not this guard's job
88+
try {
89+
// fileURLToPath requires an authority (even an empty one); `file:C:/x`
90+
// needs a slash inserted before the drive letter to parse as one.
91+
const href = isBareWindowsDrive ? dbPath.replace(/^file:/i, "file:/") : dbPath
92+
return fileURLToPath(href)
93+
} catch {
94+
return undefined
95+
}
96+
}
97+
// altimate_change end
98+
4299
/**
43100
* The store path a file-backed connection names, or a loud failure.
44101
*
@@ -69,6 +126,26 @@ export function allowsCreate(config: ConnectionConfig): boolean {
69126
return config.create === true
70127
}
71128

129+
// altimate_change start — a directory at dbPath is not a valid store either
130+
/**
131+
* Whether `path` names a store file that actually exists. `fs.existsSync`
132+
* alone is also true for a directory at that path, which the driver would
133+
* then fail to open with a confusing engine-level error instead of this
134+
* guard's clear one — a config that names a directory (a typo, or a path one
135+
* level off) should be reported the same way a missing file is.
136+
*/
137+
function existsAsFile(path: string): boolean {
138+
if (!fs.existsSync(path)) return false
139+
try {
140+
return !fs.statSync(path).isDirectory()
141+
} catch {
142+
// A stat failure (e.g. a race with a delete, or a permissions error)
143+
// is not this guard's to diagnose — let the driver's own open surface it.
144+
return true
145+
}
146+
}
147+
// altimate_change end
148+
72149
/**
73150
* Throw unless the store is safe to open: it already exists, the caller opted
74151
* in to creating it, or the path is not a local file at all.
@@ -85,8 +162,24 @@ export function assertStoreExists(
85162
allowCreate: boolean = allowsCreate(config),
86163
): void {
87164
if (allowCreate) return
165+
// altimate_change start — existence-check an absolute `file:` URI too;
166+
// `isLocalFilePath` deliberately excludes `file:` (see its own comment), so
167+
// without this branch every `file:` store — including absolute ones that
168+
// name one unambiguous on-disk location regardless of cwd — skipped the
169+
// guard entirely and a missing absolute file: store opened silently empty,
170+
// the exact bug class this guard exists to catch.
171+
const fileUriPath = absoluteFileUriPath(dbPath)
172+
if (fileUriPath !== undefined) {
173+
if (existsAsFile(fileUriPath)) return
174+
throw new Error(
175+
`${engine} database file not found: "${dbPath}" (resolved to "${fileUriPath}"). ` +
176+
`Opening a warehouse connection never creates the database — an empty store would answer every query with no rows. ` +
177+
`Check the "path" in your connection config, or pass "create": true if this store is meant to be created.`,
178+
)
179+
}
180+
// altimate_change end
88181
if (!isLocalFilePath(dbPath)) return
89-
if (fs.existsSync(dbPath)) return
182+
if (existsAsFile(dbPath)) return
90183
throw new Error(
91184
`${engine} database file not found: "${dbPath}". ` +
92185
`Opening a warehouse connection never creates the database — an empty store would answer every query with no rows. ` +

‎packages/drivers/test/file-store-guard.test.ts‎

Lines changed: 72 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ import { Database } from "bun:sqlite"
1515
import * as fs from "fs"
1616
import * as os from "os"
1717
import * as path from "path"
18-
import { allowsCreate, assertStoreExists, isLocalFilePath } from "../src/file-store"
18+
import { allowsCreate, assertStoreExists, absoluteFileUriPath, isLocalFilePath } from "../src/file-store"
1919

2020
const CANARY_TABLE = "zorbulax_ledger"
2121

@@ -57,6 +57,19 @@ describe("isLocalFilePath", () => {
5757
expect(isLocalFilePath(":memory")).toBe(true)
5858
expect(isLocalFilePath(":foo")).toBe(true)
5959
})
60+
61+
// altimate_change start — regression: an ordinary filename that merely
62+
// contains a colon must not be misread as a remote scheme
63+
test("a local filename shaped like `scheme:name` (no `//`) is still a local file", () => {
64+
// The exclusion used to match ANY "2+ letter prefix + colon", so a file
65+
// literally named "data:warehouse.duckdb" was misclassified as a remote
66+
// target and silently skipped both path resolution and the existence
67+
// guard. Only a real `scheme://` URI or one of the specific non-slash
68+
// DuckDB extension schemes (md:, motherduck:, ducklake:) should be excluded.
69+
expect(isLocalFilePath("data:warehouse.duckdb")).toBe(true)
70+
expect(isLocalFilePath("foo:warehouse.db")).toBe(true)
71+
})
72+
// altimate_change end
6073
})
6174

6275
describe("assertStoreExists", () => {
@@ -90,6 +103,64 @@ describe("assertStoreExists", () => {
90103
expect(allowsCreate({ type: "duckdb", create: "true" })).toBe(false)
91104
expect(allowsCreate({ type: "duckdb" })).toBe(false)
92105
})
106+
107+
// altimate_change start — an absolute `file:` URI is not a "local file" by
108+
// isLocalFilePath (see its own comment), but it still names one unambiguous
109+
// on-disk location and must be existence-checked, or a missing absolute
110+
// file: store opens silently empty — the exact bug class this guard exists
111+
// to catch.
112+
test("throws for a missing absolute `file:` URI", () => {
113+
const missing = path.join(tmp(), "absent.duckdb")
114+
const uri = `file://${missing}`
115+
expect(() => assertStoreExists({ type: "duckdb" }, uri, "DuckDB")).toThrow("not found")
116+
expect(fs.existsSync(missing)).toBe(false)
117+
})
118+
119+
test("passes for an existing absolute `file:` URI", () => {
120+
const dir = tmp()
121+
const present = path.join(dir, "present.duckdb")
122+
fs.writeFileSync(present, "")
123+
const uri = `file://${present}`
124+
expect(() => assertStoreExists({ type: "duckdb" }, uri, "DuckDB")).not.toThrow()
125+
})
126+
127+
test("does not existence-check a relative `file:` URI (tracked separately as #1209)", () => {
128+
// resolveStorePaths leaves relative file: URIs untouched, so guarding
129+
// existence here would check against whatever the process cwd happens to
130+
// be — the exact cwd-following bug this PR removes for plain paths. That
131+
// stays out of scope; absoluteFileUriPath returns undefined for it and
132+
// the guard falls through isLocalFilePath's own file: exclusion.
133+
expect(() => assertStoreExists({ type: "duckdb" }, "file:relative/warehouse.duckdb", "DuckDB")).not.toThrow()
134+
})
135+
// altimate_change end
136+
137+
// altimate_change start — a directory at dbPath is not a valid store
138+
test("throws when dbPath names a directory, not a file", () => {
139+
const dir = tmp()
140+
expect(() => assertStoreExists({ type: "duckdb" }, dir, "DuckDB")).toThrow("not found")
141+
})
142+
// altimate_change end
143+
})
144+
145+
describe("absoluteFileUriPath", () => {
146+
test("resolves an absolute `file://` URI to its filesystem path", () => {
147+
expect(absoluteFileUriPath("file:///var/data/warehouse.duckdb")).toBe("/var/data/warehouse.duckdb")
148+
})
149+
150+
test("returns undefined for a relative `file:` URI", () => {
151+
expect(absoluteFileUriPath("file:relative/warehouse.duckdb")).toBeUndefined()
152+
})
153+
154+
test("returns undefined for the in-memory/temporary forms", () => {
155+
expect(absoluteFileUriPath("file:")).toBeUndefined()
156+
expect(absoluteFileUriPath("file::memory:")).toBeUndefined()
157+
expect(absoluteFileUriPath("file:test.db?mode=memory")).toBeUndefined()
158+
})
159+
160+
test("returns undefined for a non-`file:` path", () => {
161+
expect(absoluteFileUriPath("/var/data/warehouse.duckdb")).toBeUndefined()
162+
expect(absoluteFileUriPath(":memory:")).toBeUndefined()
163+
})
93164
})
94165

95166
describe("DuckDB driver create-on-open", () => {

‎packages/opencode/src/altimate/native/connections/registry.ts‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,12 +114,20 @@ function resolveStorePaths(
114114
for (const [name, config] of Object.entries(entries)) {
115115
const storePath = config?.path
116116
const type = typeof config?.type === "string" ? config.type.toLowerCase() : ""
117+
// altimate_change start — recognize a Windows-absolute path even when this
118+
// process runs on POSIX. path.isAbsolute() is platform-bound: on macOS/Linux
119+
// it does not recognize `C:\...`, so a config shared or migrated from a
120+
// Windows machine had its already-absolute path re-mangled through
121+
// path.resolve(baseDir, ...) below, producing something like
122+
// "/project/C:\Users\me\warehouse.duckdb" instead of being left untouched.
117123
if (
118124
!FILE_STORE_TYPES.has(type) ||
119125
typeof storePath !== "string" ||
120126
!isLocalFilePath(storePath) ||
121-
path.isAbsolute(storePath)
127+
path.isAbsolute(storePath) ||
128+
path.win32.isAbsolute(storePath)
122129
) {
130+
// altimate_change end
123131
resolved[name] = config
124132
continue
125133
}

‎packages/opencode/src/altimate/tools/sql-execute.ts‎

Lines changed: 38 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,13 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
8080
const responseError = normalizeError((result as SqlExecuteResult & { error?: unknown }).error)
8181
if (responseError !== undefined) {
8282
const msg = responseError.trim() || "SQL execution failed."
83+
// altimate_change start — fingerprint a failed execution too. The
84+
// fingerprint used to be emitted only on the success path below, so a
85+
// failed query (this error branch, and the thrown-exception catch
86+
// further down) never got fingerprinted at all — biasing the sql
87+
// structure telemetry away from exactly the queries most worth seeing.
88+
emitSqlFingerprint(args.query, ctx.sessionID)
89+
// altimate_change end
8390
// altimate_change — annotate this failure too, same as the catch block below:
8491
// a fail-open notice that only rides on success under-counts fail-open in
8592
// precisely the cases most likely to fail.
@@ -93,26 +100,7 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
93100

94101
let output = formatResult(result)
95102
// altimate_change start — emit SQL structure fingerprint telemetry
96-
try {
97-
const fp = computeSqlFingerprint(args.query)
98-
if (fp) {
99-
Telemetry.track({
100-
type: "sql_fingerprint",
101-
timestamp: Date.now(),
102-
session_id: ctx.sessionID,
103-
statement_types: JSON.stringify(fp.statement_types),
104-
categories: JSON.stringify(fp.categories),
105-
table_count: fp.table_count,
106-
function_count: fp.function_count,
107-
has_subqueries: fp.has_subqueries,
108-
has_aggregation: fp.has_aggregation,
109-
has_window_functions: fp.has_window_functions,
110-
node_count: fp.node_count,
111-
})
112-
}
113-
} catch {
114-
// Fingerprinting must never break query execution
115-
}
103+
emitSqlFingerprint(args.query, ctx.sessionID)
116104
// altimate_change end
117105
// altimate_change start — progressive disclosure suggestions
118106
const suggestion = PostConnectSuggestions.getProgressiveSuggestion("sql_execute")
@@ -134,6 +122,10 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
134122
})
135123
} catch (e) {
136124
const msg = e instanceof Error ? e.message : String(e)
125+
// altimate_change start — fingerprint a thrown-exception failure too (see the
126+
// matching comment on the result-error branch above)
127+
emitSqlFingerprint(args.query, ctx.sessionID)
128+
// altimate_change end
137129
// altimate_change — annotate the failure too. A fail-open notice that only rides
138130
// on success is worse than none: the reason vanishes exactly when the call went
139131
// wrong, and the `precedence` marker under-counts fail-open in precisely the
@@ -147,6 +139,32 @@ export const SqlExecuteTool = Tool.define("sql_execute", {
147139
},
148140
})
149141

142+
// altimate_change start — emit SQL structure fingerprint telemetry on every
143+
// execution outcome (success, a result-shaped error, or a thrown exception),
144+
// not only on success. Extracted so all three call sites stay in sync.
145+
function emitSqlFingerprint(query: string, sessionID: string): void {
146+
try {
147+
const fp = computeSqlFingerprint(query)
148+
if (!fp) return
149+
Telemetry.track({
150+
type: "sql_fingerprint",
151+
timestamp: Date.now(),
152+
session_id: sessionID,
153+
statement_types: JSON.stringify(fp.statement_types),
154+
categories: JSON.stringify(fp.categories),
155+
table_count: fp.table_count,
156+
function_count: fp.function_count,
157+
has_subqueries: fp.has_subqueries,
158+
has_aggregation: fp.has_aggregation,
159+
has_window_functions: fp.has_window_functions,
160+
node_count: fp.node_count,
161+
})
162+
} catch {
163+
// Fingerprinting must never break query execution
164+
}
165+
}
166+
// altimate_change end
167+
150168
// altimate_change start — pre-execution SQL validation via cached schema
151169
const CACHE_TTL_MS = 24 * 60 * 60 * 1000 // 24 hours
152170
// High ceiling so large warehouses aren't arbitrarily truncated; we emit

‎packages/opencode/src/altimate/tools/warehouse-add.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ export const WarehouseAddTool = Tool.define("warehouse_add", {
3535
- sqlite: path (file path), create (optional, default false)
3636
- clickhouse: host, port, database, user, password, protocol (http/https), connection_string, request_timeout, tls_ca_cert, tls_cert, tls_key, clickhouse_settings
3737
- trino: host, port, catalog, schema, user, password, protocol (http/https), connection_string, access_token, extra_headers
38-
File-backed stores (duckdb, sqlite): the store must already exist — connecting never creates it, because an empty database answers every query with no rows. Pass "create": true only when the store is meant to be created. A relative "path" is resolved against the directory of the config that declares it (the global config resolves against ~/.altimate-code), never against the current working directory; prefer an absolute path.
38+
File-backed stores (duckdb, sqlite): the store must already exist — connecting never creates it, because an empty database answers every query with no rows. Pass "create": true only when the store is meant to be created. A relative "path" is resolved once, at add-time, against the project root (not the current working directory) and persisted absolute — even though this tool always writes to the global config file, "add" resolves against the project you're in right now, not against ~/.altimate-code; prefer an absolute path to avoid relying on that.
3939
Snowflake auth examples: (1) Password: {"type":"snowflake","account":"xy12345","user":"admin","password":"secret","warehouse":"WH","database":"db"}. (2) Key-pair: {"type":"snowflake","account":"xy12345","user":"admin","private_key_path":"/path/rsa_key.p8","warehouse":"WH","database":"db"}. (3) OAuth: {"type":"snowflake","account":"xy12345","authenticator":"oauth","token":"<token>","warehouse":"WH","database":"db"}. (4) SSO: {"type":"snowflake","account":"xy12345","user":"admin","authenticator":"externalbrowser","warehouse":"WH","database":"db"}.
4040
IMPORTANT: For private key file paths, always use "private_key_path" (not "private_key").`,
4141
),

0 commit comments

Comments
 (0)