Skip to content

Commit d13f070

Browse files
anandgupta42claude
andcommitted
fix(drivers): keep harvested roots inside the trust boundary, and make the install lock hold
Review findings on #1201, grouped by what they actually break. **Harvested roots crossed a deliberate security boundary.** `driverSearchRoots()` refuses project and ancestor `node_modules` because importing a workspace- controlled SDK during a warehouse read/test bypasses the permission boundary and can expose resolved credentials. `searchRootsFromError()` mined any absolute path an error quoted and prepended it, so an ambient failure naming a project-local `node_modules` routed straight around that invariant. Harvested roots are now filtered through the same exclusion. **They also preempted the managed installation.** Prepending meant a stale or broken copy the runtime happened to name won over the driver we installed, which inverts the documented "managed install dir comes first" ordering. Trusted roots now come first and harvested roots are appended, which still recovers the original failure: a harvested root is reached whenever the roots ahead of it resolve nothing. **Windows never harvested anything.** The extraction regex accepts `C:/…` and `C:\…`, but the `node_modules` marker is built from `path.sep`, so a forward-slash path could never match a backslash marker. `enclosingNodeModulesRoot` normalises separators, and takes `sep` as a parameter so the Windows behaviour is tested from a POSIX host rather than asserted. `repairCwdPrefixedPath` now also handles the two Windows concatenation shapes, which have no separator to carry. **The lock silently degraded to unlocked in exactly the cold-start case it exists for.** `<dir>.lock` sits beside the managed directory, and on a fresh machine nothing has created the XDG data directory yet — `performInstall` is the first thing that does, and it runs after the lock attempt. The non-recursive `mkdir` failed ENOENT, took the "cannot lock" branch, and dropped every concurrent CLI into an unlocked install. The parent is now created first. **Stale-lock recovery could admit two installers.** Two processes could both judge a lock stale, and deleting by pathname let the loser delete the winner's fresh lock. Claiming is now a `rename`, which exactly one process can win. **A lock could be released out from under its successor.** An owner whose lock was broken as stale would, on the way out, delete the lock a peer had since taken. Release now only removes a lock still carrying its own token. **Age aged out live installs.** `isStaleLock` applied the age check unconditionally, so a live same-host owner running a slow native build past `staleAfterMs` had its lock broken and a peer entered — reintroducing the very ENOTEMPTY race. Where liveness is decidable (owner on this host) it is now the only signal; age applies only where it cannot be (no readable owner, or another host sharing a home directory). **The lock path was outside the approved permission pattern.** The tool asks for `external_directory` on `<dir>/*`, but the lock is the sibling `<dir>.lock` — so creating, writing and removing it mutated an external path the user never approved. It is now included in the request. **Two unrelated cwd faults on the failure path.** `process.cwd()` throws once the working directory is removed, and both `loadDiagnostics` and — pre-existing — `resolveOptionalPackage`'s `createRequire` anchor called it unguarded while a driver failure was being formatted, replacing the real diagnosis with an unrelated `uv_cwd` ENOENT. Both are guarded; the anchor never needed cwd, since resolution is driven by the explicit `paths`. Tests. The two end-to-end resolution tests used the specifier `duckdb`, which the repository's own `packages/drivers/node_modules` can satisfy no matter what the harvesting code does — they passed while proving nothing, and would have kept passing after the reordering above. They now use specifiers that exist nowhere but the tree the test builds, and assert on an export marker, so they establish which root actually satisfied the load. The cross-process exclusion test gained a start barrier: without one, a scheduler that ran the four children serially satisfied the no-overlap assertion even with a completely broken lock. The control confirms the barriered test still bites — four unlocked children interleave completely (enter 1, enter 2, enter 3, enter 0) and the assertion rejects on the second enter. 9 repeat runs, 72/72 green. Not verified: Windows on hardware. The separator handling is now unit-tested through the `sep` parameter, but nothing here ran on Windows. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VqnuBDGkh1ZT65Ti7e6DHZ
1 parent 16ac5e5 commit d13f070

5 files changed

Lines changed: 535 additions & 73 deletions

File tree

‎packages/drivers/src/resolve.ts‎

Lines changed: 208 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -172,12 +172,83 @@ function nodeModulesUpward(start: string): string[] {
172172
* exists. That last check is what keeps a legitimately nested
173173
* `<cwd>/node_modules/…` from being mangled.
174174
*/
175-
export function repairCwdPrefixedPath(candidate: string, cwd = process.cwd()): string | undefined {
176-
if (!candidate || fs.existsSync(candidate)) return undefined
177-
if (!candidate.startsWith(cwd + path.sep)) return undefined
178-
const remainder = candidate.slice(cwd.length)
179-
if (!path.isAbsolute(remainder)) return undefined
180-
return fs.existsSync(remainder) ? remainder : undefined
175+
export function repairCwdPrefixedPath(candidate: string, cwd = safeCwd()): string | undefined {
176+
if (!candidate || !cwd || fs.existsSync(candidate)) return undefined
177+
if (!candidate.startsWith(cwd)) return undefined
178+
// POSIX concatenation yields `<cwd>/usr/…`, whose remainder carries the
179+
// separator. Windows has no separator to carry: `C:\work` + `C:\global\…`
180+
// concatenates to `C:\workC:\global\…`, and a join-shaped `C:\work\C:\global\…`
181+
// leaves a stray leading separator on the remainder. Try each shape and
182+
// accept only a remainder that is absolute and exists, so a near-miss such
183+
// as cwd `/work` against `/workspace/…` contributes nothing.
184+
const rest = candidate.slice(cwd.length)
185+
for (const remainder of rest.startsWith(path.sep) ? [rest, rest.slice(1)] : [rest]) {
186+
if (!remainder || !path.isAbsolute(remainder)) continue
187+
if (fs.existsSync(remainder)) return remainder
188+
}
189+
return undefined
190+
}
191+
192+
/**
193+
* `process.cwd()` throws ENOENT when the working directory has been removed out
194+
* from under the process. Every caller here is formatting an error or repairing
195+
* a path, where throwing would replace the driver failure the caller is trying
196+
* to report with an unrelated ENOENT — losing the actual diagnosis.
197+
*/
198+
function safeCwd(): string | undefined {
199+
try {
200+
return process.cwd()
201+
} catch {
202+
return undefined
203+
}
204+
}
205+
206+
/**
207+
* The `node_modules` content `driverSearchRoots()` deliberately refuses to
208+
* search: the working directory tree, and the project and ancestor
209+
* `node_modules` above it.
210+
*/
211+
function workspaceScope(): { cwd: string | undefined; ancestors: string[] } {
212+
const cwd = safeCwd()
213+
if (!cwd) return { cwd: undefined, ancestors: [] }
214+
const resolved = path.resolve(cwd)
215+
return { cwd: resolved, ancestors: nodeModulesUpward(resolved).map((dir) => path.resolve(dir)) }
216+
}
217+
218+
/**
219+
* Preserve order, drop repeats. A harvested root often names a directory the
220+
* inferred roots already cover, and listing it twice makes the searched-location
221+
* count in the failure message overstate where we actually looked.
222+
*/
223+
function dedupeRoots(roots: readonly string[]): string[] {
224+
return [...new Set(roots)]
225+
}
226+
227+
/**
228+
* The `node_modules` directory enclosing `candidate`, or undefined when there
229+
* is none.
230+
*
231+
* `sep` is a parameter so the Windows behaviour is testable from a POSIX host.
232+
* It matters because Windows quotes both `C:\…` and `C:/…` in errors, while the
233+
* marker is built from the platform separator — a forward-slash path would
234+
* never match a backslash marker, and the root would silently not be harvested.
235+
* Rewriting separators is length-preserving, so the slice offsets still hold.
236+
*/
237+
export function enclosingNodeModulesRoot(candidate: string, sep: string = path.sep): string | undefined {
238+
const normalized = sep === "\\" ? candidate.replace(/\//g, "\\") : candidate
239+
const marker = `${sep}node_modules${sep}`
240+
const at = normalized.lastIndexOf(marker)
241+
if (at === -1) return undefined
242+
return normalized.slice(0, at + marker.length - 1)
243+
}
244+
245+
/** True when `root` is workspace-controlled and must not be imported from. */
246+
function isWorkspaceRoot(root: string, scope: { cwd: string | undefined; ancestors: string[] }): boolean {
247+
const resolved = path.resolve(root)
248+
if (scope.ancestors.includes(resolved)) return true
249+
if (!scope.cwd) return false
250+
const rel = path.relative(scope.cwd, resolved)
251+
return rel === "" || (!rel.startsWith("..") && !path.isAbsolute(rel))
181252
}
182253

183254
/**
@@ -189,9 +260,16 @@ export function repairCwdPrefixedPath(candidate: string, cwd = process.cwd()): s
189260
* infer, so it is worth searching — after repairing a concatenated working
190261
* directory, which is the failure this exists for. Returns roots only when they
191262
* exist on disk, so a nonsense path contributes nothing.
263+
*
264+
* Workspace-controlled roots are never returned. `driverSearchRoots()` refuses
265+
* project and ancestor `node_modules` because importing a matching SDK during a
266+
* warehouse read/test would bypass the permission boundary and can expose
267+
* resolved credentials; mining a path out of an error message must not become a
268+
* way around that invariant.
192269
*/
193270
export function searchRootsFromError(error: unknown): string[] {
194271
const message = error instanceof Error ? error.message : String(error)
272+
const scope = workspaceScope()
195273
const roots: string[] = []
196274
// Absolute paths the runtime quoted, POSIX or Windows.
197275
for (const match of message.matchAll(/['"`]((?:\/|[A-Za-z]:[\\/])[^'"`\n]+)['"`]/g)) {
@@ -200,11 +278,10 @@ export function searchRootsFromError(error: unknown): string[] {
200278
for (const candidate of [named, repairCwdPrefixedPath(named)]) {
201279
if (!candidate) continue
202280
// Walk back to the enclosing node_modules directory.
203-
const marker = `${path.sep}node_modules${path.sep}`
204-
const at = candidate.lastIndexOf(marker)
205-
if (at === -1) continue
206-
const root = candidate.slice(0, at + marker.length - 1)
207-
if (isDirectory(root) && !roots.includes(root)) roots.push(root)
281+
const root = enclosingNodeModulesRoot(candidate)
282+
if (!root || !isDirectory(root)) continue
283+
if (isWorkspaceRoot(root, scope)) continue
284+
if (!roots.includes(root)) roots.push(root)
208285
}
209286
}
210287
return roots
@@ -278,7 +355,13 @@ export function packageNameOf(specifier: string): string {
278355
*/
279356
export function resolveOptionalPackage(specifier: string, roots = driverSearchRoots()): string | undefined {
280357
const pkg = packageNameOf(specifier)
281-
const require = createRequire(pathToFileURL(path.join(process.cwd(), "noop.js")).href)
358+
// The anchor only has to be some absolute file URL — resolution is driven by
359+
// the explicit `paths` below, not by this base. So it must not be the one
360+
// thing that can throw: process.cwd() raises ENOENT when the working
361+
// directory has been removed, and letting that escape here replaces every
362+
// driver diagnosis with an unrelated uv_cwd error.
363+
const anchor = safeCwd() ?? os.tmpdir()
364+
const require = createRequire(pathToFileURL(path.join(anchor, "noop.js")).href)
282365

283366
for (const root of roots) {
284367
const pkgDir = path.join(root, pkg)
@@ -371,11 +454,18 @@ export async function loadOptionalDriver(
371454
return await importer(specifier)
372455
} catch (ambientError) {
373456
const ambientBroken = !isModuleNotFound(ambientError, specifier)
374-
// Search the location the runtime itself named first. When ambient
375-
// resolution fails it frequently quotes the absolute path it was reaching
376-
// for, and that beats anything we can infer — including the case where it
377-
// concatenated the working directory onto an already-absolute path.
378-
const roots = [...searchRootsFromError(ambientError), ...driverSearchRoots()]
457+
// Trusted roots first, then the location the runtime itself named. When
458+
// ambient resolution fails it frequently quotes the absolute path it was
459+
// reaching for — including the case where it concatenated the working
460+
// directory onto an already-absolute path — and that is the only evidence
461+
// available when nothing else resolves. But it is evidence about wherever
462+
// the runtime happened to point, which may be a stale or broken copy, so it
463+
// must not preempt the managed installation: `driverSearchRoots()` puts the
464+
// driver we installed first precisely so it wins over a stale copy
465+
// elsewhere. Appending keeps that order and still recovers the failure this
466+
// exists for, because a harvested root is reached whenever the roots ahead
467+
// of it resolve nothing.
468+
const roots = dedupeRoots([...driverSearchRoots(), ...searchRootsFromError(ambientError)])
379469
const resolved = resolveOptionalPackage(specifier, roots)
380470

381471
if (!resolved) {
@@ -467,7 +557,11 @@ function ambientLoadFailure(driver: DriverName, error: unknown, detail: string):
467557
* — which is answerable on the spot, and only from inside the failing process.
468558
*/
469559
function loadDiagnostics(error: unknown): string {
470-
const lines = [`cwd=${process.cwd()}`, `execPath=${process.execPath}`]
560+
// A deleted working directory makes process.cwd() throw. This runs while a
561+
// driver failure is being formatted, so letting that escape would replace the
562+
// fault the reader needs with an unrelated ENOENT from the reporting path.
563+
const cwd = safeCwd()
564+
const lines = [`cwd=${cwd ?? "<unavailable>"}`, `execPath=${process.execPath}`]
471565
const message = error instanceof Error ? error.message : String(error)
472566
for (const match of message.matchAll(/['"`]((?:\/|[A-Za-z]:[\\/])[^'"`\n]+)['"`]/g)) {
473567
const named = match[1]
@@ -503,10 +597,10 @@ export async function loadOptionalPackage(specifier: string): Promise<any | unde
503597
return await import(/* @vite-ignore */ specifier)
504598
} catch (ambientError) {
505599
if (!isModuleNotFound(ambientError, specifier)) throw ambientError
506-
const resolved = resolveOptionalPackage(specifier, [
507-
...searchRootsFromError(ambientError),
508-
...driverSearchRoots(),
509-
])
600+
const resolved = resolveOptionalPackage(
601+
specifier,
602+
dedupeRoots([...driverSearchRoots(), ...searchRootsFromError(ambientError)]),
603+
)
510604
if (!resolved) return undefined
511605
return await import(/* @vite-ignore */ pathToFileURL(resolved).href)
512606
}
@@ -878,7 +972,7 @@ const installsInFlight = new Map<string, Promise<InstallResult>>()
878972
* The lock lives beside the install directory rather than inside it so npm
879973
* never sees it as stray package content.
880974
*/
881-
function installLockPath(dir: string): string {
975+
export function installLockPath(dir: string): string {
882976
return `${dir.replace(/[\\/]+$/, "")}.lock`
883977
}
884978

@@ -887,6 +981,8 @@ interface LockHolder {
887981
hostname: string
888982
/** Absent when the owner file is malformed; age then falls back to mtime. */
889983
startedAt?: number
984+
/** Identifies one acquisition, so a holder only ever releases its own lock. */
985+
token?: string
890986
}
891987

892988
function readLockHolder(lockDir: string): LockHolder | undefined {
@@ -900,22 +996,33 @@ function readLockHolder(lockDir: string): LockHolder | undefined {
900996
pid: holder.pid,
901997
hostname: typeof holder.hostname === "string" ? holder.hostname : "",
902998
startedAt: typeof holder.startedAt === "number" ? holder.startedAt : undefined,
999+
token: typeof holder.token === "string" ? holder.token : undefined,
9031000
}
9041001
} catch {
9051002
return undefined
9061003
}
9071004
}
9081005

9091006
/**
910-
* True when a lock cannot belong to a live install any more: its owner is gone,
911-
* or it has outlived any plausible npm run. Both checks are needed — a killed
912-
* process leaves no signal beyond its absence, and a lock from another host
913-
* (shared home directory) can only be judged by age.
1007+
* True when a lock cannot belong to a live install any more.
1008+
*
1009+
* The two signals are not interchangeable, and which one applies depends on
1010+
* whether liveness is decidable at all:
1011+
*
1012+
* - **Owner on this host.** Liveness is decidable, so it is the only thing that
1013+
* counts. Age must *not* also apply here: npm can legitimately run longer
1014+
* than any duration we pick — a native build such as `oracledb` or `duckdb`,
1015+
* or a caller that raised its own install timeout — and breaking a live
1016+
* owner's lock puts two `npm install` runs over the same tree, which is the
1017+
* exact corruption this lock exists to prevent.
1018+
* - **No readable owner, or an owner on another host** (a shared home
1019+
* directory). Liveness cannot be established, so age is the only signal
1020+
* available and the lock ages out.
9141021
*/
9151022
function isStaleLock(lockDir: string, holder: LockHolder | undefined, maxAgeMs: number): boolean {
916-
if (holder && holder.hostname === os.hostname() && !processExists(holder.pid)) return true
1023+
if (holder && holder.hostname === os.hostname()) return !processExists(holder.pid)
9171024
const startedAt = holder?.startedAt
918-
if (typeof startedAt === "number" && Date.now() - startedAt > maxAgeMs) return true
1025+
if (typeof startedAt === "number") return Date.now() - startedAt > maxAgeMs
9191026
try {
9201027
return Date.now() - fs.statSync(lockDir).mtimeMs > maxAgeMs
9211028
} catch {
@@ -924,6 +1031,56 @@ function isStaleLock(lockDir: string, holder: LockHolder | undefined, maxAgeMs:
9241031
}
9251032
}
9261033

1034+
/**
1035+
* Take ownership of a lock judged stale, atomically, and remove it.
1036+
*
1037+
* Two processes can both judge the same lock stale. If each simply deleted the
1038+
* pathname, the first would delete the dead lock and acquire a fresh one, and
1039+
* the second would then delete *that* live lock and acquire its own — putting
1040+
* both inside the critical section, which is the failure the lock exists to
1041+
* prevent. `rename` is atomic: exactly one process can move a given directory,
1042+
* and only that process goes on to delete it. The loser's rename fails and it
1043+
* simply retries against whatever state now exists.
1044+
*/
1045+
function claimStaleLock(lockDir: string): void {
1046+
const claimed = `${lockDir}.stale-${process.pid}-${Date.now()}`
1047+
try {
1048+
fs.renameSync(lockDir, claimed)
1049+
} catch {
1050+
// Another process claimed it first, or the owner released it. Either way
1051+
// there is nothing of ours to clean up; retry the acquire.
1052+
return
1053+
}
1054+
try {
1055+
fs.rmSync(claimed, { recursive: true, force: true })
1056+
} catch {
1057+
// The rename already made the lock unreachable, so a leftover directory
1058+
// beside it costs nothing but disk.
1059+
}
1060+
}
1061+
1062+
/**
1063+
* Release a lock this process acquired, but only while it is still ours.
1064+
*
1065+
* A lock we hold can be broken as stale and re-taken by a peer while `fn` is
1066+
* still running — an install that outlives `staleAfterMs` on a machine whose
1067+
* owner record is unreadable, say. Removing it by pathname would then delete
1068+
* the successor's live lock and admit a third process. The token is written
1069+
* when the lock is taken, so a mismatch means the directory is somebody else's.
1070+
*/
1071+
function releaseInstallLock(lockDir: string, token: string): void {
1072+
const holder = readLockHolder(lockDir)
1073+
// An unreadable owner file leaves nothing to compare against; the lock is
1074+
// most likely still ours (we wrote it) and leaking it would wedge every later
1075+
// install behind a lock nobody releases, so remove it.
1076+
if (holder?.token !== undefined && holder.token !== token) return
1077+
try {
1078+
fs.rmSync(lockDir, { recursive: true, force: true })
1079+
} catch {
1080+
// Leaving it behind is safe: the next contender ages it out as stale.
1081+
}
1082+
}
1083+
9271084
/**
9281085
* Run `fn` while holding an exclusive lock on `dir`, across processes.
9291086
*
@@ -942,8 +1099,22 @@ export async function withInstallLock<T>(
9421099
const staleAfterMs = options.staleAfterMs ?? 300_000
9431100
const pollMs = options.pollMs ?? 100
9441101
const deadline = Date.now() + timeoutMs
1102+
const token = `${process.pid}-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 10)}`
9451103
let acquired = false
9461104

1105+
// The lock's parent must exist before the atomic mkdir can land. On a cold
1106+
// machine nothing has created the XDG data directory yet — `performInstall`
1107+
// is the first thing that does, and it runs *after* this — so a non-recursive
1108+
// mkdir would fail ENOENT, take the "cannot lock" branch, and drop every
1109+
// caller straight to an unlocked install. That is precisely the cold-start
1110+
// stampede this lock exists to prevent, so create the parent first.
1111+
try {
1112+
fs.mkdirSync(path.dirname(lockDir), { recursive: true })
1113+
} catch {
1114+
// Genuinely unwritable. The acquire below then fails too and we proceed
1115+
// unlocked, which is the documented degradation.
1116+
}
1117+
9471118
for (;;) {
9481119
try {
9491120
fs.mkdirSync(lockDir, { recursive: false })
@@ -955,11 +1126,7 @@ export async function withInstallLock<T>(
9551126
const code = e && typeof e === "object" && "code" in e ? (e as { code?: unknown }).code : undefined
9561127
if (code !== "EEXIST") break
9571128
if (isStaleLock(lockDir, readLockHolder(lockDir), staleAfterMs)) {
958-
try {
959-
fs.rmSync(lockDir, { recursive: true, force: true })
960-
} catch {
961-
// Another process won the cleanup; fall through and retry.
962-
}
1129+
claimStaleLock(lockDir)
9631130
continue
9641131
}
9651132
if (Date.now() >= deadline) break
@@ -971,7 +1138,12 @@ export async function withInstallLock<T>(
9711138
try {
9721139
fs.writeFileSync(
9731140
path.join(lockDir, "owner.json"),
974-
JSON.stringify({ pid: process.pid, hostname: os.hostname(), startedAt: Date.now() } satisfies LockHolder),
1141+
JSON.stringify({
1142+
pid: process.pid,
1143+
hostname: os.hostname(),
1144+
startedAt: Date.now(),
1145+
token,
1146+
} satisfies LockHolder),
9751147
)
9761148
} catch {
9771149
// Diagnostics only — the lock is the directory, not the file in it.
@@ -981,13 +1153,7 @@ export async function withInstallLock<T>(
9811153
try {
9821154
return await fn(acquired)
9831155
} finally {
984-
if (acquired) {
985-
try {
986-
fs.rmSync(lockDir, { recursive: true, force: true })
987-
} catch {
988-
// Leaving it behind is safe: the next contender ages it out as stale.
989-
}
990-
}
1156+
if (acquired) releaseInstallLock(lockDir, token)
9911157
}
9921158
}
9931159

0 commit comments

Comments
 (0)