Skip to content

Commit f1a79df

Browse files
test(utils): load the production module in the cross-process child and gate the waits
Four dispositions from comment 4543975553 on #2368, one per reviewed hunk. Root 4538510000 (Major, fileLock.test.ts:756). The child was plain CommonJS that restated the artefact rule and hardcoded a 31000 window, because plain node cannot load fileLock.ts: its extensionless ../core/task-persistence/persistenceTrace import fails with ERR_MODULE_NOT_FOUND. The parent now registers the repository's own tsx loader and runs the child as node --input-type=module --import <tsx> -e <script>, and the child imports the production module and calls its acquireFileLock, so the staleness window, the lock key and the artefact spelling come from fileLock.ts rather than from the script, and the artefact the child reports is the one the same canonicalLeafSpelling call produces. Measured before writing this: with the loader armed the child imports the module, acquires, is refused ELOCKED by a foreign holder, and releases clean. The negative control that must name the artefact by the caller's own spelling still drives the library directly, and it now takes its window from LOCK_STALE_MS instead of a literal. Root 4540025576 (Minor, :1329). holdMs 1500 made the child let go on a timer while this process still had assertions to run. The child now holds until this process signals through a done file, and the ceiling is a backstop; the trace names which of the two let go, and the row asserts ASSERTIONS-DONE present and HOLD-EXPIRED absent, so a pass bought by the timeout reddens. Root 4540025584 (Minor, :1464). The waiting-writer rows gated on the child's ARTEFACT line, which proves the child started, not that it attempted and was refused. The child emits ATTEMPT <n> and REFUSED <code> for every attempt: it drives its own attempts through the module at a constant gap, because a refusal inside the library's internal retries is not observable from outside them, and both waiter rows now wait for a REFUSED ELOCKED line before this process touches the file. Root 4540025589 (Minor, :1687). The third arrival was probed at target, which names the first holder's artefact, while the comment above it claims a refusal at the new target on the artefact the second lock armed. The probe goes through nextTarget now. Controls, all run by me in this worktree. Old head 9d21fc3: prettier --check exit 0 on the three touched paths, eslint --prune-suppressions --max-warnings=0 exit 0 with no suppression entry naming any of them, vitest unit leg on this file 29 passed 8 skipped (37) in 19.83s, tsc --noEmit exit 0. After the change, same commands, same worktree: prettier --write and --check exit 0, eslint exit 0, vitest 29 passed 8 skipped (37) in 17.47s, tsc --noEmit exit 0. At the main tip 53cb5b5 the same commands ran in a worktree whose node_modules is a junction to this one; its prettier --check is exit 0 and its vitest leg passes 2 of 2 rows, its eslint stops at eslint.config.mjs importing an export this worktree's packages/config-eslint does not have, and its tsc reports 22 errors in provider, webview and shared files and none in fileLock.ts or its test; both reds name workspace packages resolved through the junction, and the file it holds is a 30-line ancestor rather than this file. The mutation preflight for the unit delta is recorded in the push note rather than here. Dispositions 2 and 3 run on this box; disposition 4's row needs a file symlink and fs.symlinkSync returns EPERM here, so it skips, and the claim it now asserts was measured instead: two real files, two holders, third arrival at the new target ELOCKED, the new target admitted once its own holder released while the first target is still refused. Co-Authored-By: Claude <[email protected]>
1 parent 9d21fc3 commit f1a79df

1 file changed

Lines changed: 187 additions & 78 deletions

File tree

‎src/utils/__tests__/fileLock.test.ts‎

Lines changed: 187 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import * as fs from "fs"
33
import * as os from "os"
44
import { createRequire } from "module"
55
import * as path from "path"
6+
import { fileURLToPath, pathToFileURL } from "url"
67

78
import * as lockfile from "proper-lockfile"
89

@@ -32,6 +33,16 @@ vi.mock("fs", async (importOriginal) => {
3233
return { ...actual, statSync }
3334
})
3435

36+
/**
37+
* The paths the cross-process child needs in order to load the production module instead of imitating
38+
* it. This file and the module sit in the same package, so resolving from this file's own address gives
39+
* absolute paths without guessing the layout, and the loader is the repository's own `tsx`, resolved the
40+
* same way the child's copy of `proper-lockfile` is resolved below.
41+
*/
42+
const productionModulePath = path.resolve(fileURLToPath(new URL("../fileLock.ts", import.meta.url)))
43+
const leafSpellingModulePath = path.resolve(fileURLToPath(new URL("../lockLeafSpelling.ts", import.meta.url)))
44+
const productionLoaderUrl = pathToFileURL(createRequire(import.meta.url).resolve("tsx")).href
45+
3546
describe("file lock protocol", () => {
3647
beforeEach(() => vi.clearAllMocks())
3748

@@ -708,13 +719,17 @@ describe("identity keeps inodes above 2^53 distinct", () => {
708719
* two spellings that produced two artefacts would let both writers in and no assertion in this file
709720
* would notice.
710721
*
711-
* The child is plain CommonJS inside a template literal rather than a TypeScript import: loading
712-
* `fileLock.ts` from a child fails with ERR_MODULE_NOT_FOUND on the extensionless
713-
* `../core/task-persistence/persistenceTrace` import, which is the module's own import chain, and the
714-
* child therefore restates the one rule it needs - the artefact naming of `lockLeafSpelling`, naming
715-
* the artefact by the spelling the volume reports for the leaf and falling back to the caller's own
716-
* spelling where the volume has none - while the key comes from the module's `canonicalLockKey`,
717-
* computed by the parent and passed in, so the child decides nothing else for itself.
722+
* The child is a script inside a template literal that the parent runs through the repository's own
723+
* TypeScript loader, `node --input-type=module --import <tsx> -e <script>`, and it then imports the
724+
* production `fileLock.ts` and calls its `acquireFileLock`. Plain node cannot load that file from a
725+
* child: it fails with ERR_MODULE_NOT_FOUND on the extensionless
726+
* `../core/task-persistence/persistenceTrace` import, which is the module's own import chain. Measured
727+
* at this head with the loader registered, the import resolves, so the staleness window, the retry
728+
* shape, the lock key and the artefact spelling all come from the module rather than from this script,
729+
* and the artefact the child reports is the one produced by the same `canonicalLeafSpelling` call the
730+
* module makes. The negative control below deliberately opts out with `DSH_LOCK_NAIVE`, drives the
731+
* library itself and names the artefact by the caller's own spelling, which is the behaviour that
732+
* control exists to show.
718733
*
719734
* The handshake is a trace file the child appends to synchronously, not its stdout. Node writes to a
720735
* piped stdout asynchronously, including on Windows, so a child that ends the process can lose the
@@ -724,18 +739,22 @@ describe("identity keeps inodes above 2^53 distinct", () => {
724739
* The exported `lock` of proper-lockfile 4.1.2 is its promise wrapper, measured at this head in
725740
* `index.js`: `module.exports = lock` over `async function lock(file, options)`, which takes no callback,
726741
* so a callback passed as a third argument is dropped without a word and the child would sit forever
727-
* holding a lock it never told anyone about. The child therefore chains `.then` and `.catch`, which is
728-
* the same shape `fileLock.ts` itself awaits.
742+
* holding a lock it never told anyone about. The child therefore awaits it, which is the same shape
743+
* `fileLock.ts` itself awaits, and it awaits the module's `acquireFileLock` the same way.
729744
*/
730-
const HOLDER_SCRIPT = `const fs = require("fs")
731-
const path = require("path")
732-
const lockfile = require(process.env.DSH_LOCK_MODULE)
745+
const HOLDER_SCRIPT = `import { pathToFileURL } from "url"
746+
const fs = await import("fs")
747+
const path = await import("path")
748+
const { createRequire } = await import("module")
733749
const spelling = process.env.DSH_LOCK_SPELLING
734750
const trace = process.env.DSH_LOCK_TRACE
735751
const holdMs = Number(process.env.DSH_LOCK_HOLD_MS || 0)
736752
const retries = Number(process.env.DSH_LOCK_RETRIES || 0)
753+
const retryMs = Number(process.env.DSH_LOCK_RETRY_MS || 250)
754+
const doneFile = process.env.DSH_LOCK_DONE_FILE || ""
737755
const baseMode = process.env.DSH_LOCK_BASE
738756
const writeKey = process.env.DSH_LOCK_WRITE_KEY || ""
757+
const naive = process.env.DSH_LOCK_NAIVE === "1"
739758
function traceLine(line) {
740759
fs.appendFileSync(trace, line + "\\n", "utf8")
741760
}
@@ -746,83 +765,134 @@ function readBody(target) {
746765
return {}
747766
}
748767
}
749-
let artefact = (function (target) {
750-
let leaf
751-
try {
752-
leaf = path.basename(fs.realpathSync.native(target))
753-
} catch {
754-
leaf = path.basename(target)
755-
}
756-
return path.join(path.dirname(target), leaf) + ".lock"
757-
})(spelling)
758-
if (process.env.DSH_LOCK_NAIVE === "1") {
768+
function sleep(ms) {
769+
return new Promise((resolve) => setTimeout(resolve, ms))
770+
}
771+
// The production module, loaded through the loader the parent registered, so the staleness window, the
772+
// lock key and the artefact spelling are the module's and not this script's. The naive control is the
773+
// one shape that must not import it: it names the artefact by the caller's own spelling and drives the
774+
// library directly, which is the behaviour that control exists to show.
775+
const leafSpelling = naive ? null : await import(pathToFileURL(process.env.DSH_LOCK_SPELLING_MODULE_TS).href)
776+
const production = naive ? null : await import(pathToFileURL(process.env.DSH_LOCK_MODULE_TS).href)
777+
const absolute = path.resolve(spelling)
778+
let artefact
779+
if (naive) {
759780
artefact = path.join(path.dirname(spelling), path.basename(spelling)) + ".lock"
781+
} else {
782+
const canonical = leafSpelling.canonicalLeafSpelling(absolute)
783+
artefact = path.join(path.dirname(canonical), path.basename(canonical)) + ".lock"
760784
}
761785
traceLine("ARTEFACT " + artefact)
762786
const staleBase = baseMode === "before" ? readBody(spelling) : null
763787
if (baseMode === "before") {
764788
traceLine("BASE-BEFORE " + JSON.stringify(staleBase))
765789
}
766-
lockfile.lock(process.env.DSH_LOCK_KEY, { lockfilePath: artefact, stale: 31000, update: 10000, realpath: false, retries: retries })
767-
.then((release) => {
768-
if (typeof release !== "function") {
769-
traceLine("OUTCOME failed no-release-function")
790+
async function naiveAcquire() {
791+
const requireFromChild = createRequire(path.join(path.dirname(trace), "holder.cjs"))
792+
const lockfile = requireFromChild(process.env.DSH_LOCK_MODULE)
793+
const releaseNaive = await lockfile.lock(process.env.DSH_LOCK_KEY, {
794+
lockfilePath: artefact,
795+
stale: Number(process.env.DSH_LOCK_STALE_MS),
796+
update: 10000,
797+
realpath: false,
798+
retries: 0,
799+
})
800+
if (typeof releaseNaive !== "function") {
801+
const notARelease = new Error("the library resolved to something other than a release function")
802+
notARelease.code = "no-release-function"
803+
throw notARelease
804+
}
805+
return releaseNaive
806+
}
807+
let release = null
808+
let attempt = 0
809+
let refusal = ""
810+
while (release === null) {
811+
attempt += 1
812+
traceLine("ATTEMPT " + attempt)
813+
try {
814+
release = naive ? await naiveAcquire() : await production.acquireFileLock(absolute, { immediate: true })
815+
} catch (error) {
816+
refusal = String(error && error.code)
817+
traceLine("REFUSED " + refusal)
818+
if (attempt > retries) {
819+
traceLine("OUTCOME failed " + refusal)
770820
process.exitCode = 1
771-
return
772-
}
773-
traceLine("OUTCOME held")
774-
if (writeKey) {
775-
const base = baseMode === "before" ? staleBase : readBody(spelling)
776-
traceLine(baseMode === "before" ? "BASE-USED " + JSON.stringify(base) : "BASE-UNDER-LOCK " + JSON.stringify(base))
777-
const merged = Object.assign({}, base)
778-
merged[writeKey] = 1
779-
fs.writeFileSync(spelling, JSON.stringify(merged), "utf8")
780-
traceLine("WROTE " + JSON.stringify(readBody(spelling)))
781-
}
782-
const letGo = () => {
783-
release().then(() => {
784-
traceLine("RELEASED")
785-
process.exitCode = 0
786-
}).catch((releaseError) => {
787-
traceLine("RELEASE-FAILED " + String(releaseError.code))
788-
process.exitCode = 1
789-
})
821+
break
790822
}
791-
if (holdMs > 0) {
792-
setTimeout(letGo, holdMs)
793-
return
823+
await sleep(retryMs)
824+
}
825+
}
826+
if (release !== null) {
827+
traceLine("OUTCOME held")
828+
if (writeKey) {
829+
const base = baseMode === "before" ? staleBase : readBody(spelling)
830+
traceLine(baseMode === "before" ? "BASE-USED " + JSON.stringify(base) : "BASE-UNDER-LOCK " + JSON.stringify(base))
831+
const merged = Object.assign({}, base)
832+
merged[writeKey] = 1
833+
fs.writeFileSync(spelling, JSON.stringify(merged), "utf8")
834+
traceLine("WROTE " + JSON.stringify(readBody(spelling)))
835+
}
836+
const startedAt = Date.now()
837+
if (holdMs > 0 || doneFile !== "") {
838+
traceLine("HOLDING " + String(holdMs))
839+
// Either the parent signals that its assertions are finished, or the ceiling it was handed runs
840+
// out, and the trace names which of the two let go.
841+
for (;;) {
842+
if (doneFile !== "" && fs.existsSync(doneFile)) {
843+
traceLine("ASSERTIONS-DONE " + String(Date.now() - startedAt))
844+
break
845+
}
846+
if (holdMs > 0 && Date.now() - startedAt >= holdMs) {
847+
traceLine("HOLD-EXPIRED " + String(Date.now() - startedAt))
848+
break
849+
}
850+
await sleep(25)
794851
}
795-
letGo()
796-
})
797-
.catch((error) => {
798-
traceLine("OUTCOME failed " + String(error.code))
852+
}
853+
try {
854+
await release()
855+
traceLine("RELEASED")
856+
} catch (releaseError) {
857+
traceLine("RELEASE-FAILED " + String(releaseError.code))
799858
process.exitCode = 1
800-
})`
801-
859+
}
860+
}`
802861
interface HolderHandle {
803862
pid: number | undefined
804863
tracePath: string
864+
donePath: string
805865
childOutput: () => string
866+
/** Tell the child its lock may go; it waits for this file to appear before releasing. */
867+
signalRelease: () => void
806868
waitForExit: () => Promise<number | null>
807869
}
808870

809871
interface HolderOptions {
810-
/** Milliseconds to hold before releasing; overrides the mode default. */
872+
/** Milliseconds to hold before releasing; overrides the mode default, and with waitForAssertions
873+
* it is the ceiling the child falls back on rather than the moment it lets go. */
811874
holdMs?: number
812875
/** A key the child merges into the file under its own lock; empty means no write. */
813876
writeKey?: string
814877
/** Where the child reads the base it merges into: after admission, or before it ever tried. */
815878
baseMode?: "under" | "before"
816-
/** Retry budget the child hands the library, so it can be the writer that has to wait. */
879+
/** Attempts the child makes before reporting the lock refused, so it can be the writer that has to wait. */
817880
retries?: number
881+
/** Milliseconds between the child's attempts. */
882+
retryMs?: number
883+
/** Keep the lock until this process calls signalRelease, so the assertions run under the other
884+
* process's lock rather than under a timeout this row happened to win. */
885+
waitForAssertions?: boolean
818886
}
819887

820888
/**
821889
* Start the other process. `mode: "try"` asks it to take the lock and end, which is what a contention
822890
* assertion needs; `mode: "hold"` makes it hold for six seconds so the suite can assert from this
823891
* process that a live foreign holder is enough to keep the module out. The options turn the same child
824892
* into the two shapes the persistence and lifecycle rows need: a writer that lands while others queue,
825-
* and a writer that has to queue behind a lock this process is holding.
893+
* and a writer that has to queue behind a lock this process is holding. With waitForAssertions the
894+
* child keeps its own lock until this process signals, so an assertion in a row below runs while the
895+
* foreign holder still holds rather than after a timer released it.
826896
*/
827897
function startLockHolder(
828898
spelling: string,
@@ -834,24 +904,36 @@ function startLockHolder(
834904
options: HolderOptions = {},
835905
): HolderHandle {
836906
const tracePath = path.join(directory, traceName)
837-
const child = spawn(process.execPath, ["-e", HOLDER_SCRIPT], {
838-
cwd: process.cwd(),
839-
stdio: ["ignore", "pipe", "pipe"],
840-
windowsHide: true,
841-
env: {
842-
...process.env,
843-
NO_COLOR: "1",
844-
DSH_LOCK_MODULE: lockModule,
845-
DSH_LOCK_SPELLING: spelling,
846-
DSH_LOCK_TRACE: tracePath,
847-
DSH_LOCK_HOLD_MS: String(options.holdMs ?? (mode === "hold" ? 6000 : 0)),
848-
DSH_LOCK_RETRIES: String(options.retries ?? 0),
849-
DSH_LOCK_BASE: options.baseMode ?? "under",
850-
DSH_LOCK_WRITE_KEY: options.writeKey ?? "",
851-
DSH_LOCK_NAIVE: naive ? "1" : "0",
852-
DSH_LOCK_KEY: canonicalLockKey(path.resolve(spelling)),
907+
const donePath = path.join(directory, traceName + ".done")
908+
// The loader argument is what makes the production import resolvable in a child at all; without it
909+
// the module's extensionless import chain fails before the child can call anything.
910+
const child = spawn(
911+
process.execPath,
912+
["--input-type=module", "--import", productionLoaderUrl, "-e", HOLDER_SCRIPT],
913+
{
914+
cwd: process.cwd(),
915+
stdio: ["ignore", "pipe", "pipe"],
916+
windowsHide: true,
917+
env: {
918+
...process.env,
919+
NO_COLOR: "1",
920+
DSH_LOCK_MODULE: lockModule,
921+
DSH_LOCK_MODULE_TS: productionModulePath,
922+
DSH_LOCK_SPELLING_MODULE_TS: leafSpellingModulePath,
923+
DSH_LOCK_STALE_MS: String(LOCK_STALE_MS),
924+
DSH_LOCK_SPELLING: spelling,
925+
DSH_LOCK_TRACE: tracePath,
926+
DSH_LOCK_HOLD_MS: String(options.holdMs ?? (mode === "hold" ? 6000 : 0)),
927+
DSH_LOCK_RETRIES: String(options.retries ?? 0),
928+
DSH_LOCK_RETRY_MS: String(options.retryMs ?? 250),
929+
DSH_LOCK_BASE: options.baseMode ?? "under",
930+
DSH_LOCK_WRITE_KEY: options.writeKey ?? "",
931+
DSH_LOCK_NAIVE: naive ? "1" : "0",
932+
DSH_LOCK_KEY: canonicalLockKey(path.resolve(spelling)),
933+
DSH_LOCK_DONE_FILE: options.waitForAssertions === true ? donePath : "",
934+
},
853935
},
854-
})
936+
)
855937
let output = ""
856938
const note = (chunk: string): void => {
857939
output += chunk
@@ -867,7 +949,11 @@ function startLockHolder(
867949
return {
868950
pid: child.pid,
869951
tracePath,
952+
donePath,
870953
childOutput: () => (output.length > 400 ? output.slice(-400) : output),
954+
signalRelease: () => {
955+
fs.writeFileSync(donePath, "", "utf8")
956+
},
871957
waitForExit: () =>
872958
new Promise((resolve) => {
873959
if (child.exitCode !== null) {
@@ -1326,17 +1412,28 @@ describe("an unrelated lock and a held leaf coexist while the alias contends", (
13261412
fs.writeFileSync(sibling, '{"pre":1}', "utf8")
13271413
const releaseLeaf = await acquireFileLock(target, { immediate: true })
13281414
const holder = startLockHolder(sibling, "hold", lockModule, directory, nextTraceName(), false, {
1329-
holdMs: 1500,
1415+
holdMs: 6000,
1416+
waitForAssertions: true,
13301417
})
13311418
await readHolderTrace(holder.tracePath, "ARTEFACT", 15000, holder.childOutput)
13321419
const outcomeLine = await readHolderTrace(holder.tracePath, "OUTCOME", 15000, holder.childOutput)
13331420
expect(traceOutcome(outcomeLine)).toBe("held")
1421+
// Armed, and the trace says so: the child holds until this process tells it otherwise.
1422+
await readHolderTrace(holder.tracePath, "HOLDING", 15000, holder.childOutput)
13341423
expect(artefactsOf(directory)).toEqual([
13351424
"state-sidecar-leaf-for-sibling-admission.json.lock",
13361425
"task-history-leaf-for-sibling-admission.json.lock",
13371426
])
1427+
// The release is this process's to start. Everything above ran while the other process still held
1428+
// the sibling lock, and the trace has to name the signal rather than the six-second ceiling the
1429+
// child was handed as a backstop: a row that passed because the child timed out would leave
1430+
// HOLD-EXPIRED in the trace and reddens here.
1431+
holder.signalRelease()
13381432
expect(await holder.waitForExit()).toBe(0)
1339-
expect(fs.readFileSync(holder.tracePath, "utf8").split("\n")).toContain("RELEASED")
1433+
const traceLines = fs.readFileSync(holder.tracePath, "utf8").split("\n")
1434+
expect(traceLines).toContain("RELEASED")
1435+
expect(traceLines.some((line) => line.startsWith("ASSERTIONS-DONE "))).toBe(true)
1436+
expect(traceLines.some((line) => line.startsWith("HOLD-EXPIRED "))).toBe(false)
13401437
await releaseLeaf()
13411438
expect(artefactsOf(directory)).toEqual([])
13421439
})
@@ -1462,6 +1559,10 @@ describe("a writer that had to wait merges into what landed while it waited", ()
14621559
retries: 5,
14631560
})
14641561
await readHolderTrace(waiter.tracePath, "ARTEFACT", 15000, waiter.childOutput)
1562+
// Observed rather than assumed: the waiter attempted while this process still held the lock and
1563+
// was refused, so the row cannot pass with a writer that merely ran after the write below.
1564+
const refusedLine = await readHolderTrace(waiter.tracePath, "REFUSED", 15000, waiter.childOutput)
1565+
expect(refusedLine).toBe("REFUSED ELOCKED")
14651566
// Measured rather than assumed: this process wrote while the other one had still not been
14661567
// admitted, which is what makes it a waiting writer at all rather than a second writer that
14671568
// merely ran later.
@@ -1492,6 +1593,10 @@ describe("a writer that had to wait merges into what landed while it waited", ()
14921593
})
14931594
const beforeLine = await readHolderTrace(waiter.tracePath, "BASE-BEFORE", 15000, waiter.childOutput)
14941595
expect(tracePayload(beforeLine, "BASE-BEFORE")).toEqual({ pre: 1 })
1596+
// The base was read before the attempt, and the attempt then happened and was refused; both are
1597+
// in the trace before this process touches the file, which is the barrier this row was missing.
1598+
const refusedLine = await readHolderTrace(waiter.tracePath, "REFUSED", 15000, waiter.childOutput)
1599+
expect(refusedLine).toBe("REFUSED ELOCKED")
14951600
expect(fs.readFileSync(waiter.tracePath, "utf8")).not.toContain("OUTCOME held")
14961601
writeMergedKey(target, "writerA")
14971602
await release()
@@ -1683,8 +1788,12 @@ describe("a replaced referent lets go both locks and the renewal timer", () => {
16831788
expect(artefactsOf(directory)).toEqual(["history.json.lock", "history.json.next.lock"])
16841789
expect(ledger.outstanding()).toBe(2)
16851790
// The second lock is a directory on a volume, not bookkeeping: a third arrival at the new target
1686-
// itself is refused on the artefact the second lock armed.
1687-
await expect(acquireFileLock(target, { immediate: true })).rejects.toMatchObject({ code: "ELOCKED" })
1791+
// itself is refused on the artefact the second lock armed. Probed through `nextTarget`, because
1792+
// `target` names the first holder's artefact, and asking there measures a refusal this row is not
1793+
// claiming.
1794+
await expect(acquireFileLock(nextTarget, { immediate: true })).rejects.toMatchObject({
1795+
code: "ELOCKED",
1796+
})
16881797
// The record names the second arrival, so the second release is the one that lands.
16891798
expect(await releaseQuietly(releaseSecond)).toBeNull()
16901799
releaseSecond = null

0 commit comments

Comments
 (0)