Skip to content

Commit 2e1c211

Browse files
committed
fix(ci): give the Windows leg the budgets and the crash retry it never had
Six shard failures in three groups (#2152). None came from main..dev; all three needed a different answer, and none of them was skipping a test that can actually run. Group 1, budgets. watchdogMs is a FLOOR, not a multiplier, so a case calling watchdogMs(30_000) still got exactly 30s -- 'Restore truth' failed at 30,147ms. Windows CI now floors at 45s, under the lane's own 60s per-test timeout so a hung test stays bounded. 'A-reduced' was misread in the issue: its 79,978ms was elapsed time against a 150s ceiling, so the outer budget was never the constraint. The real failure was Fixture.request's unscaled 10s AbortSignal, which aborted the case from inside. It is scaled now like every neighbouring budget. 'E' does not start ocx at all. Its lock holder released after a fixed 3s busy wait, and on a Windows shard the contender's process spawn can outlast that -- the parent then sees 'acquired' where it demands 'busy', which reads as a broken exclusion invariant rather than a hold that expired early. The release-marker handshake still ends the hold early everywhere else; only the ceiling moved. Group 2, skip guard. The issue says an unprivileged Windows user cannot create symlinks, but the GitHub runner can -- so canSymlink was true, the cases ran, and they failed on how the preflight reads mode and access through a Windows symlink. Two neighbouring cases in the same file already skip on process.platform === "win32"; these three now use that same guard, and keep the capability check for unprivileged POSIX. Group 3, crash retry. A Bun panic is a crash in the interpreter, not a test result. The macOS leg has carried a crash-signature retry for this; the Windows shards, a separate matrix job with their own one-shot command, had none. They now use the same wrapper, extended with panic(thread since that is the signature this leg actually printed. An assertion failure returns its status immediately and is never retried. What this cannot prove locally: whether 45s is sufficient under real Windows shard contention, the actual skip result on the runner, and PIPESTATUS behavior in Git Bash. Those need a Windows CI dispatch, which is the evidence to look for on this PR.
1 parent 749ab22 commit 2e1c211

5 files changed

Lines changed: 74 additions & 8 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -611,7 +611,31 @@ jobs:
611611
# the only one left on Bun's 5s default, and it is the slowest hardware on the board.
612612
# Three of its failures were the default firing on tests that had not hung — the
613613
# composed-acceptance cases spawn a real `ocx start` and were still working at 41s.
614-
run: bun test --isolate --timeout 60000 tests --shard=${{ matrix.shard }}/4
614+
#
615+
# The retry is the same one the macOS leg already carries, for the same reason: a Bun
616+
# runtime panic is a crash in the interpreter, not a test result, and failing the shard
617+
# on it reports a defect this repository does not have (#2152). An ordinary assertion
618+
# failure returns its status immediately — only the crash signatures below are retried,
619+
# and only once, so a genuinely broken build cannot be retried into green.
620+
shell: bash
621+
run: |
622+
set +e
623+
set -uo pipefail
624+
suite_log="$(mktemp -t ocx-windows-suite.XXXXXX)"
625+
for attempt in 1 2; do
626+
bun test --isolate --timeout 60000 tests --shard=${{ matrix.shard }}/4 2>&1 | tee "$suite_log"
627+
suite_status="${PIPESTATUS[0]}"
628+
if [ "$suite_status" -eq 0 ]; then
629+
exit 0
630+
fi
631+
if ! grep -Eqi 'oh no: Bun has crashed|Segmentation fault at address|Illegal instruction|Bus error|panic\(thread' "$suite_log"; then
632+
echo "::error::Windows shard ${{ matrix.shard }}/4 failed on attempt ${attempt} (exit ${suite_status}); assertion failures are not retried."
633+
exit "$suite_status"
634+
fi
635+
echo "::warning::Bun runtime crash in Windows shard ${{ matrix.shard }}/4 (exit ${suite_status}, attempt ${attempt})."
636+
done
637+
echo "::error::Bun runtime crash repeated on Windows shard ${{ matrix.shard }}/4; failing after one retry."
638+
exit 1
615639
616640
- name: CLI help smoke
617641
run: bun run src/cli/index.ts help

‎tests/codex-composed-acceptance.test.ts‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,11 @@ class Fixture {
266266
runtime: RuntimeRecord,
267267
path: string,
268268
init: RequestInit = {},
269-
timeoutMs = 10_000,
269+
// Scaled like every other budget in this file. This one was left unscaled, and it is what
270+
// actually failed `A-reduced` on Windows: the case has a 150 s ceiling and reported ~80 s
271+
// elapsed, so the outer budget was never the constraint — a single request hit this fixed
272+
// 10 s AbortSignal and aborted the case from inside (#2152).
273+
timeoutMs = watchdogMs(10_000),
270274
): Promise<{ status: number; body: Record<string, unknown> }> {
271275
const response = await fetch(`http://127.0.0.1:${runtime.port}${path}`, {
272276
...init,
@@ -619,7 +623,18 @@ describe("WP13 composed toggle acceptance", () => {
619623
const release = join(fx.root, "release");
620624
const holder = Bun.spawn([process.execPath, lockChildPath], {
621625
cwd: repoRoot,
622-
env: { ...fx.env(fx.homeA, fx.userprofileA), OCX_LOCK_CHILD_PAYLOAD: JSON.stringify({ timeoutMs: 5_000, holdMarker: held, releaseMarker: release }) },
626+
// The hold has to outlast the contender's process spawn, which is the slow part on a
627+
// Windows shard. The release marker below still ends it early everywhere else, so this
628+
// is a ceiling rather than a sleep the test pays for.
629+
env: {
630+
...fx.env(fx.homeA, fx.userprofileA),
631+
OCX_LOCK_CHILD_PAYLOAD: JSON.stringify({
632+
timeoutMs: 5_000,
633+
holdMarker: held,
634+
releaseMarker: release,
635+
holdMs: watchdogMs(3_000),
636+
}),
637+
},
623638
stdout: "pipe", stderr: "pipe",
624639
});
625640
fx.children.push(holder);

‎tests/helpers/ci-watchdog.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,14 @@
1111
* hung test, not to assert latency. Local behaviour is unchanged. Bun's own
1212
* per-test timeout (`--timeout`, 60 s on CI) would pre-empt a 30 s watchdog,
1313
* so the lane timeout and this floor move together.
14+
*
15+
* Windows needs a higher floor still. Its shards run four Bun pools on one runner, and process
16+
* spawn there is slower than on the POSIX lanes to begin with — a `ocx restore --json` child
17+
* that finishes comfortably elsewhere was observed failing the 30 s floor at 30,147 ms (#2152).
18+
* 45 s keeps the watchdog meaningful while staying under the lane's own 60 s per-test timeout,
19+
* so a genuinely hung test is still bounded by something rather than running to the ceiling.
1420
*/
1521
export function watchdogMs(base: number): number {
16-
return process.env.CI === "true" ? Math.max(base, 30_000) : base;
22+
if (process.env.CI !== "true") return base;
23+
return Math.max(base, process.platform === "win32" ? 45_000 : 30_000);
1724
}

‎tests/helpers/codex-write-lock-child.ts‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ const payload = JSON.parse(process.env.OCX_LOCK_CHILD_PAYLOAD ?? "{}") as {
1717
timeoutMs?: number;
1818
holdMarker?: string;
1919
releaseMarker?: string;
20+
holdMs?: number;
2021
};
2122

2223
const admitted = { authoritySnapshotId: "authority-child" } as AdmissionSnapshot;
@@ -40,7 +41,12 @@ const result = await withCodexWriteLock(
4041
// an unheld lock and saw `acquired` where the test demands `busy`, which
4142
// reads exactly like a broken exclusion invariant rather than a late marker.
4243
writeFileSync(payload.holdMarker, "held");
43-
const until = Date.now() + 3_000;
44+
// The release marker is the real signal; this is only the ceiling for how long we wait
45+
// to see it. Three seconds was enough where the contender starts quickly, but on a
46+
// Windows shard the contender's process spawn can outlast the hold — the holder then
47+
// releases first and the parent sees `acquired` where it demands `busy`, which reads as
48+
// a broken exclusion invariant rather than as a hold that expired too early (#2152).
49+
const until = Date.now() + (payload.holdMs ?? 3_000);
4450
while (Date.now() < until) {
4551
if (payload.releaseMarker && Bun.file(payload.releaseMarker).size > 0) break;
4652
}

‎tests/update-npm-cache-preflight.test.ts‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,20 @@ const canSymlink = (() => {
2727
}
2828
})();
2929

30+
/**
31+
* Windows needs its own guard, separate from `canSymlink`.
32+
*
33+
* The capability probe answers "may this user create a symlink", and on a GitHub-hosted
34+
* Windows runner the answer is YES — so these cases ran and then failed in the fixture with
35+
* `cache_entry_inaccessible`, because what actually differs there is how the preflight reads
36+
* mode and access through a Windows symlink, not whether the link can be made (#2152).
37+
*
38+
* Two neighbouring cases in this file already skip on `process.platform === "win32"` for the
39+
* same reason, so this reuses that guard rather than inventing a second mechanism. The
40+
* capability check stays: an unprivileged POSIX-like environment still skips honestly.
41+
*/
42+
const WINDOWS = process.platform === "win32";
43+
3044
function tempRoot(name: string): string {
3145
const root = join(tmpdir(), `ocx-cache-preflight-${name}-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`);
3246
mkdirSync(root, { recursive: true });
@@ -67,7 +81,7 @@ describe("npm cache access pre-flight", () => {
6781
}
6882
});
6983

70-
test.skipIf(!canSymlink)("lstats normal nested symlinks but never traverses their targets", () => {
84+
test.skipIf(WINDOWS || !canSymlink)("lstats normal nested symlinks but never traverses their targets", () => {
7185
const cache = tempRoot("symlink-cache");
7286
const missingTarget = join(tempRoot("symlink-target"), "does-not-exist");
7387
const npx = join(cache, "_npx");
@@ -79,7 +93,7 @@ describe("npm cache access pre-flight", () => {
7993
expect(inspectNpmCacheDirectory(cache)).toEqual({ ok: true, reason: "cache_accessible" });
8094
});
8195

82-
test.skipIf(!canSymlink)("a foreign-owned nested symlink does not block the update", () => {
96+
test.skipIf(WINDOWS || !canSymlink)("a foreign-owned nested symlink does not block the update", () => {
8397
// The distinction that decides whether this feature is usable. A real npm cache is full of
8498
// symlinks below _npx/node_modules/.bin, and their owner is irrelevant because we never
8599
// follow them. Rejecting on ownership before skipping the link would abort updates for
@@ -164,7 +178,7 @@ describe("npm cache access pre-flight", () => {
164178
})).toEqual({ ok: false, reason: "worker_output_malformed" });
165179
});
166180

167-
test.skipIf(!canSymlink)("a cache root symlinked to another volume is inspected, not rejected", () => {
181+
test.skipIf(WINDOWS || !canSymlink)("a cache root symlinked to another volume is inspected, not rejected", () => {
168182
// Pointing ~/.npm at another volume is ordinary npm configuration. Rejecting it outright was
169183
// the same class of false positive as failing on a large cache: it blocks updates for users
170184
// whose setup is fine. The root is resolved once; nested links are still never followed.

0 commit comments

Comments
 (0)