Skip to content

Commit dd899d0

Browse files
fix(u5a): defer the orphan verdict, propagate lock-key inspection failures, pin created ancestors (#2359)
Five live CodeRabbit rows on head fb03ecb, each with a row that fails without the fix: - the failure path threw OrphanedBackupError before removing its own staging file and staging directory, so every retry of the write left both behind; the verdict is raised after that cleanup now. - OrphanedBackupError blamed the backup step and called a complete copy partial, although the same error is raised after a successful copy whenever the commit or a directory fsync fails. The wording is phase-neutral and still names the target, the copy, the original failure and both errnos. - resolveLockKey answered every inspection failure with a lexical key built from the alias: a realpath EACCES on a path that is a symlink parks the writer on a different lock than the writer resolving the same file, and a mid-walk readlink EIO returned the half-resolved key. Only ENOENT falls into the tolerant walk now; readlink ends the walk only on ENOENT, ENOTDIR or EINVAL. - the created-ancestor fsync chain stopped at the shallowest directory this write created, so the entry that links that directory into the tree was never flushed. The pre-existing parent is synced last, deepest-first. - a parent this write created was never validated: expectedAncestorIdentities can only pin directories the caller walked, so whoever moved the old parent aside could move the new one, put a link at its old name, and the commit would land at the moved location. Created directories are pinned at mkdir and re-checked at both authorization sites. Two rows were strengthened rather than added: the Windows DACL row counted two icacls calls, which cannot distinguish restoring onto the target from restoring onto the name the rename produced - it now orders save, rename, restore and pins the restore's directory argument; the resetChain row awaited the first write before resetting and passed with resetChain() as a no-op - it now holds the first write inside its publish and races the second write against it. The notALink double in the lock-key spec rejected with no errno, which models neither platform: it now carries EINVAL on POSIX and ENOENT on Windows. Measured with the pinned node v22.23.4, dir C:/work/Zoo-Code-fork/wt-r290-2359/src, "node node_modules/vitest/vitest.mjs run --config vitest.config.ts --no-file-parallelism <file>": safeWriteText.spec with the new rows and no production change 6 failed | 77 passed (83), with the production change 83 passed (83); guardedWrite.spec 66 passed (86 tests? no) 66 passed (66); resetChain neutered (pendingChains.clear replaced by void pendingChains.clear) with the strengthened row 1 failed | 65 passed (66), with the previous row 66 passed (66); the DACL restore block moved ahead of the commit rename 1 failed | 82 skipped (83) on "expected [ 'save', 'restore', 'rename' ] to deeply equal [ 'save', 'rename', 'restore' ]". safeWriteJson.lockKey.spec 4 passed (4). Services shard (vitest.services.config.ts --coverage) 11 failed | 1689 passed | 1 skipped (1701), the 11 identical to the same command at the sibling unit's head 066f153 and located in services/marketplace, services/rules and services/code-index/processors/parser, so not this delta. eslint --max-warnings=0 on the four touched files: 0 problems, same as at head. tsc --noEmit -p .: exit 0. prettier --check: clean.
1 parent fb03ecb commit dd899d0

4 files changed

Lines changed: 339 additions & 31 deletions

File tree

‎src/core/tools/__tests__/guardedWrite.spec.ts‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,20 @@ function createMockTask(options: MockTaskOptions = {}): Task {
113113
return task as unknown as Task
114114
}
115115

116+
/** Resolve after the given delay, for races that must answer instead of hang. */
117+
function delay(ms: number): Promise<void> {
118+
return new Promise((resolve) => setTimeout(resolve, ms))
119+
}
120+
121+
/** Bounded poll: reports whether the condition became true within the limit. */
122+
async function waitUntil(condition: () => boolean, limitMs: number): Promise<boolean> {
123+
const deadline = Date.now() + limitMs
124+
while (!condition() && Date.now() < deadline) {
125+
await delay(5)
126+
}
127+
return condition()
128+
}
129+
116130
// -- Tests -------------------------------------------------------------------
117131

118132
describe("guardedWrite (S4a, epic #1375)", () => {
@@ -1292,16 +1306,46 @@ describe("guardedWrite (S4a, epic #1375)", () => {
12921306
})
12931307

12941308
describe("resetChain", () => {
1295-
it("detaches pending links so later writes start a fresh chain", async () => {
1309+
it("detaches a pending link so a later write to the same path publishes before it", async () => {
12961310
const reg = new ObservationRegistry()
12971311
reg.observe(abs("x.txt"), "v1")
12981312
mockedComputeVersionToken.mockResolvedValue("v1")
12991313
const task = createMockTask({ observationRegistry: reg })
1314+
// Hold the FIRST write inside its publish so that a link is genuinely pending at reset
1315+
// time. Awaiting the first write first, as this row used to, settles it, and the row then
1316+
// passes even when resetChain() does nothing at all.
1317+
let releaseFirst!: () => void
1318+
const holdFirst = new Promise<void>((resolve) => {
1319+
releaseFirst = resolve
1320+
})
1321+
let publishStarted = 0
1322+
mockedSafeWriteText.mockImplementation(async () => {
1323+
publishStarted++
1324+
if (publishStarted === 1) {
1325+
await holdFirst
1326+
}
1327+
return undefined
1328+
})
1329+
1330+
const first = guardedWrite(task, "x.txt", "a", "update")
1331+
// Bounded: the row must answer fast instead of waiting out the 20s test timeout when
1332+
// the publish never starts.
1333+
expect(await waitUntil(() => publishStarted === 1, 2000)).toBe(true)
13001334

1301-
await guardedWrite(task, "x.txt", "a", "update")
13021335
resetChain()
1303-
await guardedWrite(task, "x.txt", "b", "update")
1336+
const second = guardedWrite(task, "x.txt", "b", "update")
1337+
1338+
// A detached chain lets the second write publish while the first is STILL held. Left on
1339+
// the chain, the second link waits behind the first's pending publish and this race
1340+
// answers "blocked".
1341+
const outcome = await Promise.race([second.then(() => "published"), delay(200).then(() => "blocked")])
1342+
expect(outcome).toBe("published")
1343+
1344+
// Cleanup: release the held publish so this row leaves no pending link for the next one.
1345+
releaseFirst()
1346+
await first
13041347

1348+
expect(publishStarted).toBe(2)
13051349
expect(mockedSafeWriteText).toHaveBeenLastCalledWith(abs("x.txt"), "b", {
13061350
expectedResolvedPath: abs("x.txt"),
13071351
expectedAncestorIdentities: [],

‎src/services/file-safety/__tests__/safeWriteText.spec.ts‎

Lines changed: 216 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -616,17 +616,37 @@ describe("safeWriteText", () => {
616616
// ── Test 5: win32 DACL path ──────────────────────────────────────────────
617617

618618
describe("win32 DACL", () => {
619-
it.skipIf(process.platform !== "win32")(
620-
"copies target DACL onto staging file via icacls before rename on Windows",
621-
async () => {
622-
const targetPath = "/tmp/test-dir/target.txt"
623-
vi.mocked(fs.realpath).mockResolvedValue(targetPath)
624-
await safeWriteText(targetPath, "data", { platform: "win32" })
625-
626-
// icacls dump + restore were called (execFile is callback-based mock)
627-
expect(execFile).toHaveBeenCalledTimes(2)
628-
},
629-
)
619+
it("saves the target DACL before the commit and applies it to the published name after it", async () => {
620+
const targetPath = "/tmp/test-dir/target.txt"
621+
vi.mocked(fs.realpath).mockResolvedValue(targetPath)
622+
vi.mocked(fsSync.openSync).mockReturnValue(1)
623+
// The contract this row names, stated as an order: the target's DACL is captured while
624+
// the target is still the published file, and the dump is applied to the name that the
625+
// commit rename produced. Counting two icacls calls cannot tell those two apart, so the
626+
// calls are tagged and compared against the commit itself.
627+
const order: string[] = []
628+
vi.mocked(execFile).mockImplementation((_cmd, _args, _opts, cb) => {
629+
order.push(_args?.[1] === "/save" ? "save" : "restore")
630+
if (typeof cb === "function") cb(null, "", "")
631+
return fakeChild
632+
})
633+
vi.mocked(fs.rename).mockImplementation(async () => {
634+
order.push("rename")
635+
})
636+
637+
await safeWriteText(targetPath, "data", { platform: "win32" })
638+
639+
expect(order).toEqual(["save", "rename", "restore"])
640+
expect(execFile).toHaveBeenCalledTimes(2)
641+
// The dump is applied relative to the directory that holds the published name, which is
642+
// how icacls resolves the path recorded by /save - the published file, not the staging
643+
// file, is what the restored rights end up on.
644+
const restoreArgs = vi.mocked(execFile).mock.calls[1]?.[1]
645+
expect(restoreArgs?.[0]).toBe(path.dirname(targetPath))
646+
expect(restoreArgs?.[1]).toBe("/restore")
647+
// The dump is removed only after the restore, so no stale dump outlives the write.
648+
expect(fs.unlink).toHaveBeenCalledWith(expect.stringContaining("safeWriteText.acl"))
649+
})
630650

631651
it("non-win32: DACL path is unreachable when platform is not win32", async () => {
632652
const targetPath = "/tmp/test-dir/target.txt"
@@ -1861,7 +1881,7 @@ describe("durability of directories a write had to create", () => {
18611881
const inner = path.resolve("/tmp/root/mid/inner")
18621882
const mid = path.resolve("/tmp/root/mid")
18631883
const root = path.resolve("/tmp/root")
1864-
vi.mocked(fs.realpath).mockResolvedValue(path.resolve(targetPath))
1884+
vi.mocked(fs.realpath).mockImplementation(async (p: unknown) => path.resolve(String(p)))
18651885
// /tmp/root already exists, so the recursive mkdir creates /mid and /inner.
18661886
// Node reports the first directory it had to create; everything below it down
18671887
// to the target's own directory was created by this write too.
@@ -1873,8 +1893,189 @@ describe("durability of directories a write had to create", () => {
18731893
const opened = vi.mocked(fsSync.openSync).mock.calls.map((call) => path.resolve(String(call[0])))
18741894
expect(opened).toContain(inner)
18751895
expect(opened).toContain(mid)
1876-
expect(opened).not.toContain(root)
1877-
// staged temp + the target's directory + the newly created parent.
1878-
expect(vi.mocked(fsSync.fsyncSync).mock.calls.length).toBe(3)
1896+
// The pre-existing directory holding the entry that names the shallowest directory this
1897+
// write created belongs to the durability chain too: without it a crash can still drop
1898+
// /mid out of /tmp/root and take the whole subtree, published file included.
1899+
expect(opened).toContain(root)
1900+
expect(opened).not.toContain(path.resolve("/tmp"))
1901+
// staged temp + the target's own directory + each created parent + the pre-existing
1902+
// directory that links the created chain in.
1903+
expect(vi.mocked(fsSync.fsyncSync).mock.calls.length).toBe(4)
1904+
})
1905+
})
1906+
1907+
// ── Commit failure whose backup copy cannot be removed ─────────────────────────
1908+
describe("commit failure with an unremovable backup copy", () => {
1909+
beforeEach(() => {
1910+
mockDefaults()
1911+
vi.mocked(fsSync.writeSync).mockImplementation((...args: unknown[]) =>
1912+
typeof args[3] === "number" ? args[3] : 0,
1913+
)
1914+
})
1915+
1916+
/**
1917+
* Script one commit failure together with two failed backup unlinks. The existing
1918+
* commit-failure row lets the cleanup unlink SUCCEED, so nothing there reaches this
1919+
* combination: the copy is complete, the rename fails, and the cleanup is refused twice.
1920+
*/
1921+
function scriptCommitFailureWithStuckBackup(): {
1922+
targetPath: string
1923+
unlinked: () => string[]
1924+
run: () => Promise<unknown>
1925+
} {
1926+
const targetPath = "/tmp/test-dir/target.txt"
1927+
vi.mocked(fs.realpath).mockResolvedValue(targetPath)
1928+
vi.mocked(fsSync.openSync).mockReturnValue(1)
1929+
vi.mocked(fs.rename).mockRejectedValue(Object.assign(new Error("EACCES: rename refused"), { code: "EACCES" }))
1930+
const unlinked: string[] = []
1931+
vi.mocked(fs.unlink).mockImplementation(async (p: unknown) => {
1932+
const spelled = String(p)
1933+
if (spelled.includes("safeWriteText.bak_")) {
1934+
// A handle is still held on the copy: the retry meets the same EPERM.
1935+
throw Object.assign(new Error("EPERM: operation not permitted"), { code: "EPERM" })
1936+
}
1937+
unlinked.push(spelled)
1938+
return undefined
1939+
})
1940+
return {
1941+
targetPath,
1942+
unlinked: (): string[] => unlinked.slice(),
1943+
run: (): Promise<unknown> =>
1944+
safeWriteText(targetPath, "new data", {
1945+
backup: true,
1946+
platform: "linux",
1947+
}).catch((caught: unknown) => caught),
1948+
}
1949+
}
1950+
1951+
it("removes the staging file and its directory before reporting the orphaned backup", async () => {
1952+
const scenario = scriptCommitFailureWithStuckBackup()
1953+
const error = (await scenario.run()) as OrphanedBackupError
1954+
1955+
expect(error).toBeInstanceOf(OrphanedBackupError)
1956+
expect(error.name).toBe("OrphanedBackupError")
1957+
// The commit failure stays primary; the cleanup failure travels beside it.
1958+
expect(error.originalError).toBeInstanceOf(Error)
1959+
expect((error.originalError as Error).message).toBe("EACCES: rename refused")
1960+
expect(error.cause).toBe(error.originalError)
1961+
expect((error.cleanupError as { code?: string }).code).toBe("EPERM")
1962+
// Both attempts were made on the copy before the verdict was reached.
1963+
expect(
1964+
vi.mocked(fs.unlink).mock.calls.filter((call) => String(call[0]).includes("safeWriteText.bak_")).length,
1965+
).toBe(2)
1966+
// And the verdict must not be reached by skipping the staging cleanup: the unpublished
1967+
// temp file is unlinked and this write's own staging directory is removed. Throwing the
1968+
// orphan error first leaves both behind on every retry of the same write.
1969+
expect(scenario.unlinked().some((p) => p.includes("safeWriteText_"))).toBe(true)
1970+
expect(fs.rmdir).toHaveBeenCalledWith(expect.stringContaining(".file-safety-staging_"))
1971+
expect(fs.rename).toHaveBeenCalledTimes(1)
1972+
})
1973+
1974+
it("reports the orphaned copy in phase-neutral words", async () => {
1975+
const scenario = scriptCommitFailureWithStuckBackup()
1976+
const error = (await scenario.run()) as OrphanedBackupError
1977+
1978+
expect(error).toBeInstanceOf(OrphanedBackupError)
1979+
// The copy here was COMPLETE: the backup step succeeded and the commit is what failed.
1980+
// Wording that blames the backup creation, or calls the finished copy partial, points the
1981+
// reader at a step that went right.
1982+
expect(error.message).not.toContain("could not create the backup")
1983+
expect(error.message).not.toContain("partial")
1984+
expect(error.message).toContain("the write of /tmp/test-dir/target.txt failed")
1985+
expect(error.message).toContain("the copy this write made at")
1986+
expect(error.message).toContain("safeWriteText.bak_")
1987+
expect(error.message).toContain("EACCES: rename refused")
1988+
expect(error.message).toContain("EPERM")
1989+
})
1990+
})
1991+
1992+
// ── Lock-key inspection failures are not "end of the chain" ────────────────────
1993+
describe("resolveLockKey propagates inspection failures", () => {
1994+
beforeEach(() => mockDefaults())
1995+
1996+
it("does not key a symlink alias when realpath fails for a reason that is not ENOENT", async () => {
1997+
const alias = path.resolve("/tmp/linkdir/file.json")
1998+
// The alias and its referent are one file and must take ONE lock. A realpath that fails
1999+
// with EACCES says nothing about which spelling the path is, and answering with the
2000+
// alias's own key puts this writer on a different lock than the writer that resolves the
2001+
// same file a moment later - which is how a read-modify-write loses one side.
2002+
vi.mocked(fs.realpath).mockImplementation(async (p: unknown) => {
2003+
const spelled = String(p)
2004+
if (spelled === alias) {
2005+
throw Object.assign(new Error("EACCES: permission denied"), { code: "EACCES" })
2006+
}
2007+
return spelled
2008+
})
2009+
vi.mocked(fs.readlink).mockResolvedValue("file.json")
2010+
2011+
await expect(resolveLockKey(alias)).rejects.toThrow("EACCES")
2012+
})
2013+
2014+
it("does not treat a readlink failure that is not 'not a link' as the end of the chain", async () => {
2015+
const alias = path.resolve("/tmp/linkdir/file.json")
2016+
const enoent = Object.assign(new Error("ENOENT: no such file or directory"), { code: "ENOENT" })
2017+
vi.mocked(fs.realpath).mockRejectedValue(enoent)
2018+
vi.mocked(fs.lstat).mockResolvedValue(_fileStats(true))
2019+
// The first readlink answers - the path IS a link - and the walk onto the referent then
2020+
// hits an I/O error. Stopping there would hand back a half-resolved key as if it had been
2021+
// proved, and the next writer could take a different lock for the same file.
2022+
let readlinkCalls = 0
2023+
vi.mocked(fs.readlink).mockImplementation(async () => {
2024+
readlinkCalls++
2025+
if (readlinkCalls === 1) {
2026+
return "referent.json"
2027+
}
2028+
throw Object.assign(new Error("EIO: i/o error"), { code: "EIO" })
2029+
})
2030+
2031+
await expect(resolveLockKey(alias)).rejects.toThrow("EIO")
2032+
expect(readlinkCalls).toBe(2)
2033+
})
2034+
})
2035+
2036+
// ── Directories this write created are pinned before the commit ────────────────
2037+
describe("newly created ancestors are pinned before the commit", () => {
2038+
beforeEach(() => {
2039+
mockDefaults()
2040+
vi.mocked(fsSync.writeSync).mockImplementation((...args: unknown[]) =>
2041+
typeof args[3] === "number" ? args[3] : 0,
2042+
)
2043+
})
2044+
2045+
it("refuses the commit when a directory this write created is moved and its old name becomes a link", async () => {
2046+
const mid = path.resolve("/tmp/root/mid")
2047+
const outside = path.resolve("/outside/mid")
2048+
const inner = path.join(mid, "inner")
2049+
const targetPath = path.join(inner, "target.txt")
2050+
// The parent did not exist when the caller authorized this target, so it is not in
2051+
// expectedAncestorIdentities and cannot be pinned from that list. This write creates it
2052+
// (mkdir reports /tmp/root/mid as the first directory it created) and must pin what it
2053+
// created for itself.
2054+
vi.mocked(fs.mkdir).mockResolvedValue(mid)
2055+
let moved = false
2056+
vi.mocked(fs.realpath).mockImplementation(async (p: unknown) => {
2057+
const spelled = path.resolve(String(p))
2058+
return spelled === mid && moved ? outside : spelled
2059+
})
2060+
vi.mocked(fs.copyFile).mockImplementation(async () => {
2061+
// Between the creation and the commit the directory is moved and its old name is
2062+
// replaced with a link to the moved location. stat still reports the same (dev, ino)
2063+
// for it, so only the location comparison can see the move.
2064+
moved = true
2065+
})
2066+
vi.mocked(fsSync.openSync).mockReturnValue(1)
2067+
2068+
const error = await safeWriteText(targetPath, "new data", {
2069+
backup: true,
2070+
expectedResolvedPath: targetPath,
2071+
expectedAncestorIdentities: [],
2072+
platform: "linux",
2073+
}).catch((caught: unknown) => caught)
2074+
2075+
expect(error).toBeInstanceOf(AncestorReplacedError)
2076+
expect((error as AncestorReplacedError).directory).toBe(mid)
2077+
expect((error as AncestorReplacedError).message).toContain("no longer resolves to the authorized location")
2078+
// Nothing was published through the link.
2079+
expect(fs.rename).not.toHaveBeenCalled()
18792080
})
18802081
})

0 commit comments

Comments
 (0)