Skip to content

Commit e9466da

Browse files
anandgupta42claude
andcommitted
fix(drivers): stop a 2s deadline failing healthy DuckDB stores, and say so when the client is broken
`warehouse_test` failed on local DuckDB stores with `Timed out opening DuckDB database "<path>"` at 2002-2061ms, six of seven within 5ms of exactly 2000. Python opened the same 1.3MB store moments later and queried it fine, so the store was healthy and the driver's own deadline was the failure. The deadline was a hard-coded 2000ms with no way to raise it, and it was not a performance budget: DuckDB dispatches the open to the libuv threadpool, so the wait covers queueing behind every other threadpool user in the process. On a loaded machine a healthy open crosses it, and the driver then rejects and closes the handle that was about to succeed. - Default the budget to 30s, overridable per connection (`open_timeout_ms`) or by `ALTIMATE_DUCKDB_OPEN_TIMEOUT_MS`. - Fix a live path to the same symptom: the "callback fired synchronously" sentinel was `undefined`, which is exactly what a success callback invoked with no arguments passes, so such a callback was recorded and never replayed and the open hung until the deadline. The sentinel is now a symbol. - Say what actually happened. The old text named the store and read as "this file is broken", which sent investigators after the file for hours. Also, when the client is broken, stop it looking like a bad connection. `warehouse_test` reported "Connection 'x': FAILED" for a driver that will not load and for a wrong password alike. It now classifies local-client faults (`driver_missing`, `driver_open_timeout`, `store_locked`) and renders them as an unmistakable INFRASTRUCTURE FAILURE with `infrastructure: true` in metadata, so neither a model nor a reader of a transcript can score broken infrastructure as a task failure. Two more real faults found on the way: - `duckdb` was missing from `trustedDependencies`, so `bun install` never ran its `node-pre-gyp install` and `lib/binding/duckdb.node` was never fetched. Every DuckDB call on such a tree fails with "driver not installed" even though the package is present. Verified both directions in a scratch install. - `wrapDuckDBError` replaced DuckDB's lock message with a friendly summary, discarding the one actionable part — DuckDB names the PID and executable holding the conflicting lock. It now appends rather than replaces, and `onOpen` no longer flattens the error to a bare `DUCKDB_LOCKED`. Tests open a real store on disk; a mock is what missed this. They need their own `bun test` process, because four files in `test/altimate` install a top-level `mock.module("@altimateai/drivers/duckdb", ...)` and Bun evaluates every test file's top level before running any test, so a whole-directory run silently exercises a fake. Under `ALTIMATE_DUCKDB_E2E=1` a missing or mocked driver fails the run rather than skipping it, and CI gets a dedicated step. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent 7c8f098 commit e9466da

9 files changed

Lines changed: 439 additions & 25 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,18 @@ jobs:
294294
TEST_PG_PORT: "15432"
295295
TEST_PG_PASSWORD: testpass123
296296

297+
# Needs a dedicated process: four files in test/altimate install a
298+
# top-level mock.module("@altimateai/drivers/duckdb", …), and Bun
299+
# evaluates every test file's top level before running any test, so in a
300+
# whole-directory run these would silently exercise a fake. With
301+
# ALTIMATE_DUCKDB_E2E=1 a missing or mocked driver fails the step rather
302+
# than skipping it.
303+
- name: Run DuckDB store-open E2E (real store, no mocks)
304+
run: bun test test/altimate/duckdb-open-e2e.test.ts test/altimate/warehouse-test-duckdb-e2e.test.ts
305+
working-directory: packages/opencode
306+
env:
307+
ALTIMATE_DUCKDB_E2E: "1"
308+
297309
- name: Run Docker driver E2E (MySQL, SQL Server, Redshift)
298310
run: bun test test/altimate/drivers-docker-e2e.test.ts
299311
working-directory: packages/opencode

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@
126126
"printWidth": 120
127127
},
128128
"trustedDependencies": [
129+
"duckdb",
129130
"esbuild",
130131
"node-pty",
131132
"protobufjs",

‎packages/drivers/src/duckdb.ts‎

Lines changed: 68 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,34 @@
44

55
import type { ConnectionConfig, Connector, ConnectorResult, ExecuteOptions, SchemaColumn } from "./types"
66

7+
// altimate_change start — configurable, generous open budget
8+
/**
9+
* How long to wait for DuckDB to finish opening a store before giving up.
10+
*
11+
* This is a liveness guard, not a performance budget. Opening a store is
12+
* dispatched to the libuv threadpool, so the wait covers queueing behind every
13+
* other threadpool user in the process (fs, dns, crypto), not just DuckDB's own
14+
* work. A busy agent process can therefore push a healthy open well past a
15+
* second, and the previous hard-coded 2s ceiling turned that into an
16+
* unrecoverable failure on a store that was fine.
17+
*/
18+
const DEFAULT_OPEN_TIMEOUT_MS = 30_000
19+
20+
/** Read a positive, finite millisecond budget, or `undefined` if unusable. */
21+
function positiveMs(value: unknown): number | undefined {
22+
const n = typeof value === "number" ? value : Number(value)
23+
return Number.isFinite(n) && n > 0 ? n : undefined
24+
}
25+
26+
function resolveOpenTimeoutMs(config: ConnectionConfig): number {
27+
return (
28+
positiveMs(config.open_timeout_ms) ??
29+
positiveMs(globalThis.process?.env?.["ALTIMATE_DUCKDB_OPEN_TIMEOUT_MS"]) ??
30+
DEFAULT_OPEN_TIMEOUT_MS
31+
)
32+
}
33+
// altimate_change end
34+
735
export async function connect(config: ConnectionConfig): Promise<Connector> {
836
let duckdb: any
937
try {
@@ -14,6 +42,9 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
1442
}
1543

1644
const dbPath = (config.path as string) ?? ":memory:"
45+
// altimate_change start — configurable open budget
46+
const openTimeoutMs = resolveOpenTimeoutMs(config)
47+
// altimate_change end
1748
let db: any
1849
let connection: any
1950

@@ -35,10 +66,13 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
3566

3667
function wrapDuckDBError(err: Error): Error {
3768
if (isLockError(err)) {
69+
// Keep DuckDB's own text: it names the PID and executable holding the
70+
// conflicting lock, which is the only way to find the other process.
3871
return new Error(
3972
`Database "${dbPath}" is locked by another process. ` +
40-
`DuckDB does not support concurrent write access. ` +
41-
`Close other connections to this file and try again.`,
73+
`DuckDB takes an exclusive file lock, so a store already open ` +
74+
`read-write elsewhere cannot be opened again — not even read-only. ` +
75+
`Close the other connection and try again.\n${err.message || String(err)}`,
4276
)
4377
}
4478
return err
@@ -72,10 +106,15 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
72106
let timeout: ReturnType<typeof setTimeout> | undefined
73107
let instance: any
74108
// Sentinel for an open callback that fired synchronously (before
75-
// `instance` was assigned): `undefined` = not yet fired, `null` =
76-
// fired with success, `Error` = fired with failure. Replayed once
77-
// `instance` exists.
78-
let pendingOpen: Error | null | undefined
109+
// `instance` was assigned), replayed once `instance` exists.
110+
//
111+
// This MUST be a value the callback can never supply. It used to be
112+
// `undefined`, which is exactly what a success callback invoked with
113+
// no arguments passes — so such a callback was recorded and then
114+
// never replayed, the promise never settled, and the open failed on
115+
// the deadline below with a timeout message that named nothing.
116+
const NOT_FIRED = Symbol("duckdb-open-not-fired")
117+
let pendingOpen: Error | null | typeof NOT_FIRED = NOT_FIRED
79118
const opts = accessMode ? { access_mode: accessMode } : undefined
80119
const closeQuietly = () => {
81120
try {
@@ -84,9 +123,12 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
84123
// best-effort cleanup of a half-open handle
85124
}
86125
}
87-
const onOpen = (err: Error | null) => {
126+
const onOpen = (err?: Error | null) => {
127+
// Normalise a zero-argument success callback to `null` so it is
128+
// never confused with "has not fired yet".
129+
const outcome = err ?? null
88130
if (!instance) {
89-
pendingOpen = err
131+
pendingOpen = outcome
90132
return
91133
}
92134
if (resolved) {
@@ -95,32 +137,37 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
95137
}
96138
resolved = true
97139
if (timeout) clearTimeout(timeout)
98-
if (err) {
140+
if (outcome) {
99141
// Open failed — release the half-open handle so it doesn't leak.
142+
// Reject with DuckDB's own error: callers classify it with
143+
// isLockError(), and its text names the conflicting process.
100144
closeQuietly()
101-
const msg = err.message || String(err)
102-
if (msg.toLowerCase().includes("locked") || msg.includes("SQLITE_BUSY") || msg.includes("DUCKDB_LOCKED")) {
103-
reject(new Error("DUCKDB_LOCKED"))
104-
} else {
105-
reject(err)
106-
}
145+
reject(outcome)
107146
} else {
108147
resolve(instance)
109148
}
110149
}
111150
instance = opts
112151
? new duckdb.Database(dbPath, opts, onOpen)
113152
: new duckdb.Database(dbPath, onOpen)
114-
// Bun: native callback may not fire; fall back after 2s. Arm the timer
115-
// BEFORE replaying a synchronous callback so a sync resolve/reject can
116-
// actually clear it (otherwise it lingers ~2s and delays process exit).
153+
// Liveness guard against an open callback that never fires. Arm the
154+
// timer BEFORE replaying a synchronous callback so a sync
155+
// resolve/reject can actually clear it (otherwise it lingers and
156+
// delays process exit).
117157
timeout = setTimeout(() => {
118158
if (!resolved) {
119159
resolved = true
120-
reject(new Error(`Timed out opening DuckDB database "${dbPath}"`))
160+
reject(
161+
new Error(
162+
`DuckDB store "${dbPath}" did not finish opening within ${openTimeoutMs}ms. ` +
163+
`This is a client-side deadline in the DuckDB driver, not a fault in the store ` +
164+
`— the open may simply be queued behind other work in this process. ` +
165+
`Raise it with ALTIMATE_DUCKDB_OPEN_TIMEOUT_MS if the machine is loaded.`,
166+
),
167+
)
121168
}
122-
}, 2000)
123-
if (pendingOpen !== undefined) onOpen(pendingOpen)
169+
}, openTimeoutMs)
170+
if (pendingOpen !== NOT_FIRED) onOpen(pendingOpen)
124171
})
125172

126173
// altimate_change start — honour an explicit read-only connection.

‎packages/drivers/test/driver-security.test.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,10 @@ describe("DuckDB driver", () => {
5454
expect.unreachable("Should have thrown")
5555
} catch (e: any) {
5656
expect(e.message).toContain("locked by another process")
57-
expect(e.message).toContain("does not support concurrent write access")
57+
expect(e.message).toContain("exclusive file lock")
58+
// DuckDB's own text names the PID holding the lock, so it must survive
59+
// being wrapped rather than being replaced by the friendly summary.
60+
expect(e.message).toContain("SQLITE_BUSY: database is locked")
5861
}
5962

6063
await connector.close()

‎packages/opencode/src/altimate/native/connections/registry.ts‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,12 @@ export function detectAuthMethod(config: ConnectionConfig | null | undefined): s
282282
export function categorizeConnectionError(e: unknown): string {
283283
const msg = String(e).toLowerCase()
284284
if (msg.includes("not installed") || msg.includes("cannot find module")) return "driver_missing"
285+
// altimate_change start — categories for local-client faults
286+
// Checked before the generic "timeout"/"not found" rules below, which are
287+
// about the remote warehouse and would otherwise swallow these.
288+
if (msg.includes("did not finish opening")) return "driver_open_timeout"
289+
if (msg.includes("locked by another process")) return "store_locked"
290+
// altimate_change end
285291
if (msg.includes("password") || msg.includes("authentication") || msg.includes("unauthorized") || msg.includes("jwt"))
286292
return "auth_failed"
287293
if (msg.includes("timeout") || msg.includes("timed out")) return "timeout"
@@ -290,6 +296,19 @@ export function categorizeConnectionError(e: unknown): string {
290296
return "other"
291297
}
292298

299+
// altimate_change start — distinguish infrastructure faults from config faults
300+
/**
301+
* Categories where the local client is broken, not the connection's config and
302+
* not the remote warehouse. A caller cannot fix these by correcting
303+
* credentials, and a harness must not score them as a task failure.
304+
*/
305+
const INFRASTRUCTURE_CATEGORIES = new Set(["driver_missing", "driver_open_timeout", "store_locked"])
306+
307+
export function isInfrastructureFailure(category: string): boolean {
308+
return INFRASTRUCTURE_CATEGORIES.has(category)
309+
}
310+
// altimate_change end
311+
293312
// ---------------------------------------------------------------------------
294313
// Public API
295314
// ---------------------------------------------------------------------------
@@ -405,7 +424,9 @@ export function list(): { warehouses: WarehouseInfo[] } {
405424
}
406425

407426
/** Test a connection by running a simple query. */
408-
export async function test(name: string): Promise<{ connected: boolean; error?: string }> {
427+
export async function test(
428+
name: string,
429+
): Promise<{ connected: boolean; error?: string; error_category?: string; infrastructure?: boolean }> {
409430
try {
410431
const connector = await get(name)
411432
const config = configs.get(name)
@@ -422,7 +443,15 @@ export async function test(name: string): Promise<{ connected: boolean; error?:
422443
}
423444
return { connected: true }
424445
} catch (e) {
425-
return { connected: false, error: String(e) }
446+
// altimate_change start — report *why* the test failed, not just that it did
447+
const category = categorizeConnectionError(e)
448+
return {
449+
connected: false,
450+
error: String(e),
451+
error_category: category,
452+
infrastructure: isInfrastructureFailure(category),
453+
}
454+
// altimate_change end
426455
}
427456
}
428457

‎packages/opencode/src/altimate/native/types.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,17 @@ export interface WarehouseTestParams {
329329
export interface WarehouseTestResult {
330330
connected: boolean
331331
error?: string
332+
// altimate_change start — distinguish infrastructure faults from config faults
333+
/** Coarse cause, from `categorizeConnectionError` (e.g. `driver_missing`, `auth_failed`). */
334+
error_category?: string
335+
/**
336+
* True when the failure is in the local client — a driver that will not load,
337+
* an open that never completed — rather than in the connection's
338+
* configuration or the remote warehouse. These are not the caller's fault and
339+
* are not fixable by editing the connection.
340+
*/
341+
infrastructure?: boolean
342+
// altimate_change end
332343
}
333344

334345
// --- Warehouse Management ---

‎packages/opencode/src/altimate/tools/warehouse-test.ts‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,35 @@ export const WarehouseTestTool = Tool.define("warehouse_test", {
2020
}
2121
}
2222

23+
// altimate_change start — never let a broken client look like a bad connection
24+
// A driver that will not load, or an open that never completed, is a
25+
// fault in this machine's install. Reporting it with the same wording as
26+
// a wrong password invites both the model and anyone reading the
27+
// transcript to treat broken infrastructure as a task or config failure.
28+
if (result.infrastructure) {
29+
return {
30+
title: `Connection '${args.name}': INFRASTRUCTURE FAILURE`,
31+
metadata: {
32+
connected: false,
33+
error: result.error,
34+
error_category: result.error_category,
35+
infrastructure: true,
36+
},
37+
output:
38+
`INFRASTRUCTURE FAILURE — the warehouse client on this machine is broken. ` +
39+
`This is NOT a problem with the connection's configuration, and NOT something ` +
40+
`to work around by trying a different query or a different tool.\n` +
41+
`Category: ${result.error_category}\n` +
42+
`Error: ${result.error ?? "Unknown error"}\n\n` +
43+
`Stop and report this rather than continuing — results produced after this ` +
44+
`point did not come from warehouse '${args.name}'.`,
45+
}
46+
}
47+
// altimate_change end
48+
2349
return {
2450
title: `Connection '${args.name}': FAILED`,
25-
metadata: { connected: false },
51+
metadata: { connected: false, error: result.error, error_category: result.error_category },
2652
output: `Failed to connect to warehouse '${args.name}'.\nError: ${result.error ?? "Unknown error"}`,
2753
}
2854
} catch (e) {

0 commit comments

Comments
 (0)