Skip to content

Commit 9589cc2

Browse files
sahrizviclaude
andcommitted
fix(drivers): address consensus review — additive installs, half-installed packages, azure auth, type aliases
Findings from the multi-model consensus review of #1122. Each was reproduced before being fixed. **Installing a driver deleted the previous one.** `installOptionalDriver` ran `npm install --no-save`, so npm treated every already-installed driver as extraneous and pruned it. Reproduced on npm 11.12.1: installing `mysql2` into a prefix holding `pg` printed `added 12 packages, and removed 14 packages`. A user adding a second warehouse silently lost the first — re-creating the exact defect this module exists to fix. The install now saves to the directory's manifest, which makes it genuinely additive (verified across three drivers). **A half-installed package reported as installed.** `resolveOptionalPackage` fell back to returning the package directory when `require.resolve` failed, so an empty `node_modules/pg` resolved successfully and `isDriverInstalled` was true. `warehouse_install_driver` then answered "already installed, no action taken" and the driver could never be repaired. Resolution now requires a manifest and an entry file that exists, and keeps searching later roots instead of returning a path the caller cannot import. **Azure AD auth used the pattern this PR removes.** `sqlserver.ts` still called `import("@azure/identity" as string)`, which cannot resolve inside the compiled binary, so an installed `@azure/identity` was invisible and every Azure AD login silently fell through to the az CLI. Routed through a new `loadOptionalPackage` (soft variant that returns undefined rather than throwing, since this caller has a real fallback), and declared as a non-driver external. **Six warehouse types never got a readiness note.** `DRIVER_MAP` routes 18 type strings onto 13 drivers, but `driverForWarehouseType` matched only the 12 canonical names, so a connection added as `postgresql`, `mariadb`, `mssql`, `fabric` or `mongo` skipped the check added for #61 — the silent-broken- connection case that issue is about. **Test quality.** The review mutation-tested `isModuleNotFound` by deleting it and all 22 tests still passed; its fixture was never ambiently resolvable, so the branch was unreachable. Applying the same technique to the new fixes showed the first half-installed test was also vacuous. `isModuleNotFound` and `npmInstallArgs` are now exported and pinned directly, and four mutants — always- missing predicate, `--no-save` restored, manifest check removed, bare-directory return — each fail at least one test. Tests: 172 drivers unit (was 162), 4,712 opencode, 140 Docker-backed driver e2e. Typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1 parent 66548b5 commit 9589cc2

6 files changed

Lines changed: 224 additions & 21 deletions

File tree

‎packages/drivers/src/resolve.ts‎

Lines changed: 99 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -202,15 +202,72 @@ export function resolveOptionalPackage(specifier: string, roots = driverSearchRo
202202
const require = createRequire(pathToFileURL(path.join(process.cwd(), "noop.js")).href)
203203

204204
for (const root of roots) {
205-
if (!isDirectory(path.join(root, pkg))) continue
205+
const pkgDir = path.join(root, pkg)
206+
if (!isDirectory(pkgDir)) continue
207+
// A directory without a manifest is not an installed package — an
208+
// interrupted or half-deleted install leaves one behind. Treating it as
209+
// installed made `isDriverInstalled` report true for an empty directory,
210+
// so the install path refused to run and the driver could never be repaired.
211+
if (!fs.existsSync(path.join(pkgDir, "package.json"))) continue
212+
206213
try {
207214
return require.resolve(specifier, { paths: [root] })
208215
} catch {
209-
// ESM-only packages have no require-resolvable entry. Hand back the
210-
// package directory and let the dynamic import read its export map.
211-
const dir = path.join(root, specifier)
212-
if (isDirectory(dir) || fs.existsSync(dir)) return dir
213-
return path.join(root, pkg)
216+
// ESM-only packages expose no require-resolvable entry. Read the entry
217+
// out of the manifest instead, and only accept a file that exists.
218+
const entry = entryFromManifest(pkgDir, specifier, pkg)
219+
if (entry) return entry
220+
// Nothing importable here. Keep searching the remaining roots rather
221+
// than returning a path the caller cannot import.
222+
continue
223+
}
224+
}
225+
226+
return undefined
227+
}
228+
229+
/**
230+
* Entry file for `specifier` derived from its package manifest, or undefined
231+
* when nothing resolvable exists on disk.
232+
*/
233+
function entryFromManifest(pkgDir: string, specifier: string, pkg: string): string | undefined {
234+
const subpath = specifier.slice(pkg.length).replace(/^\//, "")
235+
236+
const candidates: string[] = []
237+
if (subpath) {
238+
// A subpath such as `mysql2/promise` usually maps to a physical file.
239+
candidates.push(
240+
path.join(pkgDir, subpath),
241+
path.join(pkgDir, `${subpath}.js`),
242+
path.join(pkgDir, `${subpath}.mjs`),
243+
path.join(pkgDir, `${subpath}.cjs`),
244+
path.join(pkgDir, subpath, "index.js"),
245+
)
246+
} else {
247+
try {
248+
const manifest = JSON.parse(fs.readFileSync(path.join(pkgDir, "package.json"), "utf8"))
249+
for (const field of ["module", "main"]) {
250+
const value = manifest?.[field]
251+
if (typeof value === "string") candidates.push(path.join(pkgDir, value))
252+
}
253+
} catch {
254+
// Unreadable or malformed manifest — fall through to the index probes.
255+
}
256+
candidates.push(path.join(pkgDir, "index.js"), path.join(pkgDir, "index.mjs"), path.join(pkgDir, "index.cjs"))
257+
}
258+
259+
for (const candidate of candidates) {
260+
try {
261+
const stat = fs.statSync(candidate)
262+
if (stat.isFile()) return candidate
263+
if (stat.isDirectory()) {
264+
for (const index of ["index.js", "index.mjs", "index.cjs"]) {
265+
const nested = path.join(candidate, index)
266+
if (fs.existsSync(nested)) return nested
267+
}
268+
}
269+
} catch {
270+
// Candidate does not exist; try the next one.
214271
}
215272
}
216273

@@ -250,7 +307,7 @@ export async function loadOptionalDriver(driver: DriverName, specifier: string):
250307
}
251308

252309
/** True when `error` means the module could not be resolved, not that it failed while loading. */
253-
function isModuleNotFound(error: unknown): boolean {
310+
export function isModuleNotFound(error: unknown): boolean {
254311
const code = (error as { code?: string } | null)?.code
255312
if (code === "ERR_MODULE_NOT_FOUND" || code === "MODULE_NOT_FOUND") return true
256313
const message = error instanceof Error ? error.message : String(error)
@@ -264,6 +321,25 @@ function loadFailure(driver: DriverName, where: string, error: unknown): Error {
264321
)
265322
}
266323

324+
/**
325+
* Import an optional package that is not a warehouse driver, returning
326+
* undefined when it is unavailable.
327+
*
328+
* Same bunfs problem as the drivers — a bare specifier cannot resolve inside
329+
* the compiled binary — but these callers have a legitimate fallback and must
330+
* not be handed an exception.
331+
*/
332+
export async function loadOptionalPackage(specifier: string): Promise<any | undefined> {
333+
try {
334+
return await import(/* @vite-ignore */ specifier)
335+
} catch (ambientError) {
336+
if (!isModuleNotFound(ambientError)) throw ambientError
337+
const resolved = resolveOptionalPackage(specifier)
338+
if (!resolved) return undefined
339+
return await import(/* @vite-ignore */ pathToFileURL(resolved).href)
340+
}
341+
}
342+
267343
/** True when `driver`'s packages are all resolvable right now. */
268344
export function isDriverInstalled(driver: DriverName, roots = driverSearchRoots()): boolean {
269345
return DRIVER_PACKAGES[driver].every((pkg) => resolveOptionalPackage(pkg, roots) !== undefined)
@@ -306,11 +382,24 @@ function runNpm(args: string[], cwd: string, timeoutMs: number): Promise<{ code:
306382
})
307383
}
308384

385+
/**
386+
* npm arguments for installing `packages` into the managed driver directory.
387+
*
388+
* `--save` is required, not incidental: with `--no-save` npm treats already
389+
* installed drivers as extraneous and prunes them on the next install.
390+
*/
391+
export function npmInstallArgs(packages: readonly string[]): string[] {
392+
return ["install", "--save", "--no-audit", "--no-fund", "--loglevel=error", ...packages]
393+
}
394+
309395
/**
310396
* Install a driver's SDK into the managed driver directory.
311397
*
312-
* Installs are additive — `--no-save` against a private package.json — so
313-
* installing a second driver never removes the first.
398+
* Installs must be recorded in the directory's own package.json. With
399+
* `--no-save`, npm treats every previously installed driver as extraneous and
400+
* prunes it: installing MySQL deleted Postgres, re-creating the very bug this
401+
* module exists to fix. Verified on npm 11.12.1 —
402+
* `added 12 packages, and removed 14 packages`.
314403
*/
315404
export async function installOptionalDriver(
316405
driver: DriverName,
@@ -345,11 +434,7 @@ export async function installOptionalDriver(
345434
}
346435
}
347436

348-
const { code, output } = await runNpm(
349-
["install", "--no-save", "--no-audit", "--no-fund", "--loglevel=error", ...packages],
350-
dir,
351-
options.timeoutMs ?? 180_000,
352-
)
437+
const { code, output } = await runNpm(npmInstallArgs(packages), dir, options.timeoutMs ?? 180_000)
353438

354439
if (code === 127) {
355440
return {

‎packages/drivers/src/sqlserver.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
*/
44

55
import type { ConnectionConfig, Connector, ConnectorResult, ExecuteOptions, SchemaColumn } from "./types"
6-
import { loadOptionalDriver } from "./resolve"
6+
import { loadOptionalDriver, loadOptionalPackage } from "./resolve"
77

88
// ---------------------------------------------------------------------------
99
// Azure AD helpers — cache + resource URL resolution
@@ -162,7 +162,12 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
162162
// who don't use Azure AD don't need to install it. Typed `any` (via a non-literal
163163
// specifier) so it compiles regardless of which @azure/identity version (if any) is
164164
// installed; the runtime API is resolved from the user's installed package.
165-
const azureIdentity: any = await import("@azure/identity" as string)
165+
// Resolved through the shared optional-package loader: a bare
166+
// specifier does not resolve inside the compiled binary, so an
167+
// installed @azure/identity was invisible and every Azure AD login
168+
// silently fell through to the az CLI path.
169+
const azureIdentity: any = await loadOptionalPackage("@azure/identity")
170+
if (!azureIdentity) throw new Error("@azure/identity is not installed")
166171
const credential = new azureIdentity.DefaultAzureCredential(
167172
config.azure_client_id
168173
? { managedIdentityClientId: config.azure_client_id as string }

‎packages/drivers/test/resolve-unit.test.ts‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ import * as path from "path"
1616

1717
import {
1818
DRIVER_PACKAGES,
19+
isModuleNotFound,
20+
npmInstallArgs,
1921
DriverNotInstalledError,
2022
driverInstallDir,
2123
driverLabel,
@@ -309,3 +311,95 @@ describe("driver catalogue", () => {
309311
expect(Object.keys(DRIVER_PACKAGES).sort()).toEqual(expected)
310312
})
311313
})
314+
315+
// ---------------------------------------------------------------------------
316+
// Regression cover for the consensus-review criticals (PR #1122)
317+
// ---------------------------------------------------------------------------
318+
319+
describe("installOptionalDriver arguments", () => {
320+
test("saves to the manifest so installs are additive", () => {
321+
// Verified on npm 11.12.1: with `--no-save`, installing mysql2 into a prefix
322+
// that already had pg printed "added 12 packages, and removed 14 packages".
323+
// Every previously installed driver is pruned as extraneous, re-creating the
324+
// exact "driver not installed" bug this module exists to fix.
325+
const args = npmInstallArgs(["mysql2"])
326+
327+
expect(args).toContain("--save")
328+
expect(args).not.toContain("--no-save")
329+
})
330+
331+
test("passes every requested package through", () => {
332+
expect(npmInstallArgs(["pg", "@types/pg"]).slice(-2)).toEqual(["pg", "@types/pg"])
333+
})
334+
})
335+
336+
describe("isModuleNotFound", () => {
337+
// Pinned directly: deleting this predicate left the behavioural tests passing,
338+
// because their fixtures are not ambiently resolvable and so never reach it.
339+
test("recognises the Node resolution error code", () => {
340+
const err = Object.assign(new Error("nope"), { code: "ERR_MODULE_NOT_FOUND" })
341+
expect(isModuleNotFound(err)).toBe(true)
342+
})
343+
344+
test("recognises the CommonJS resolution error code", () => {
345+
expect(isModuleNotFound(Object.assign(new Error("nope"), { code: "MODULE_NOT_FOUND" }))).toBe(true)
346+
})
347+
348+
test("recognises the message Bun emits inside bunfs", () => {
349+
expect(isModuleNotFound(new Error("Cannot find package 'pg' from '/$bunfs/root/index.js'"))).toBe(true)
350+
expect(isModuleNotFound(new Error("Cannot find module 'mysql2/promise'"))).toBe(true)
351+
})
352+
353+
test("does NOT classify a load-time failure as missing", () => {
354+
// The distinction that matters: a package that resolves but throws while
355+
// initialising (broken native binding) must not be reported as absent.
356+
expect(isModuleNotFound(new Error("dlopen failed: wrong architecture"))).toBe(false)
357+
expect(isModuleNotFound(new TypeError("x is not a function"))).toBe(false)
358+
expect(isModuleNotFound(undefined)).toBe(false)
359+
})
360+
})
361+
362+
describe("half-installed packages", () => {
363+
test("an empty package directory does not count as installed", () => {
364+
// An interrupted or half-deleted install leaves a bare directory behind.
365+
// Counting it as installed made warehouse_install_driver answer "already
366+
// installed, no action taken", so the driver could never be repaired.
367+
const root = path.join(tmpRoot, "node_modules")
368+
fs.mkdirSync(path.join(root, "pg"), { recursive: true })
369+
370+
expect(resolveOptionalPackage("pg", [root])).toBeUndefined()
371+
expect(isDriverInstalled("postgres", [root])).toBe(false)
372+
})
373+
374+
test("a directory with a manifest but no entry file does not count as installed", () => {
375+
const root = path.join(tmpRoot, "node_modules")
376+
const dir = path.join(root, "oracledb")
377+
fs.mkdirSync(dir, { recursive: true })
378+
fs.writeFileSync(path.join(dir, "package.json"), JSON.stringify({ name: "oracledb", main: "index.js" }))
379+
380+
expect(resolveOptionalPackage("oracledb", [root])).toBeUndefined()
381+
})
382+
383+
test("a directory with no manifest is not a package, even if a subpath file exists", () => {
384+
// Subpath probing looks for physical files (mysql2/promise.js), so without
385+
// the manifest check a bare directory holding one would resolve as an
386+
// installed package.
387+
const root = path.join(tmpRoot, "node_modules")
388+
fs.mkdirSync(path.join(root, "mysql2"), { recursive: true })
389+
fs.writeFileSync(path.join(root, "mysql2", "promise.js"), "module.exports = {}")
390+
391+
expect(resolveOptionalPackage("mysql2/promise", [root])).toBeUndefined()
392+
})
393+
394+
test("keeps searching later roots when an earlier one is half-installed", () => {
395+
const broken = path.join(tmpRoot, "broken", "node_modules")
396+
fs.mkdirSync(path.join(broken, "pg"), { recursive: true })
397+
const good = path.join(tmpRoot, "good")
398+
installFakePackage(good, "pg", "module.exports = { which: 'good' }")
399+
400+
const resolved = resolveOptionalPackage("pg", [broken, path.join(good, "node_modules")])
401+
402+
expect(resolved).toBeDefined()
403+
expect(resolved!.includes(path.join("good", "node_modules"))).toBe(true)
404+
})
405+
})

‎packages/opencode/script/build.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -245,8 +245,10 @@ const optionalExternals = [
245245
"pg", "snowflake-sdk", "@google-cloud/bigquery", "@databricks/sql",
246246
"mysql2", "mssql", "oracledb", "duckdb",
247247
"mongodb", "@clickhouse/client", "trino-client",
248-
// Optional infra packages — native addons or heavy optional deps
249-
"keytar", "ssh2", "dockerode",
248+
// Optional infra packages — native addons or heavy optional deps.
249+
// @azure/identity is dynamically imported by the sqlserver driver for Azure
250+
// AD auth; it resolves through the same on-disk loader as the drivers.
251+
"keytar", "ssh2", "dockerode", "@azure/identity",
250252
]
251253

252254
const binaries: Record<string, string> = {}

‎packages/opencode/src/altimate/tools/warehouse-install-driver.ts‎

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,13 +99,30 @@ export const WarehouseInstallDriverTool = Tool.define("warehouse_install_driver"
9999
},
100100
})
101101

102+
/**
103+
* Aliases the connection registry accepts for a warehouse type.
104+
*
105+
* `DRIVER_MAP` in native/connections/registry.ts routes 18 type strings onto
106+
* 13 drivers. Matching only the 12 canonical names meant a connection added as
107+
* `postgresql`, `mariadb`, `mssql`, `fabric` or `mongo` never got a readiness
108+
* note — the exact silent-broken-connection case #61 is about.
109+
*/
110+
const DRIVER_TYPE_ALIASES: Record<string, DriverName> = {
111+
postgresql: "postgres",
112+
mariadb: "mysql",
113+
mssql: "sqlserver",
114+
fabric: "sqlserver",
115+
mongo: "mongodb",
116+
}
117+
102118
/**
103119
* Driver name for a warehouse config `type`, or undefined when the type needs
104-
* no optional SDK (sqlite ships with the runtime).
120+
* no optional SDK (sqlite ships with the runtime) or is unrecognised.
105121
*/
106122
export function driverForWarehouseType(type: string): DriverName | undefined {
107123
const normalized = type.trim().toLowerCase()
108-
return (DRIVER_NAMES as readonly string[]).includes(normalized) ? (normalized as DriverName) : undefined
124+
if ((DRIVER_NAMES as readonly string[]).includes(normalized)) return normalized as DriverName
125+
return DRIVER_TYPE_ALIASES[normalized]
109126
}
110127

111128
export { DRIVER_PACKAGES, driverInstallDir, isDriverInstalled, installOptionalDriver, driverLabel }

‎packages/opencode/test/altimate/driver-catalogue.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ const publishScriptPath = path.join(repoRoot, "packages/opencode/script/publish.
2222
const expectedPackages = [...new Set(Object.values(DRIVER_PACKAGES).flat())].sort()
2323

2424
/** Optional infra externals that are not warehouse drivers. */
25-
const NON_DRIVER_EXTERNALS = new Set(["keytar", "ssh2", "dockerode"])
25+
const NON_DRIVER_EXTERNALS = new Set(["keytar", "ssh2", "dockerode", "@azure/identity"])
2626

2727
function readBlock(file: string, startMarker: string, endMarker: string): string {
2828
const source = fs.readFileSync(file, "utf8")

0 commit comments

Comments
 (0)