Skip to content

Commit ac52281

Browse files
authored
fix(test): deterministic temp-dir cleanup across node --test lanes (#1661)
* fix(test): deterministic temp-dir cleanup across node --test lanes node --test has no global setup/teardown hook, so unlike Vitest (#1593) every node --test package.json script (maestro:conformance, mutation:test, check:affected:test, check:coverage-changed:test, check:layering, depgraph:test, check:tmpdir-leaks:test, check:contention-retry, test:fixture-cache, test:smoke(:web), test:integration:node, test:concurrency-torture) still created scratch directories against the real, unredirected os.tmpdir(), with cleanup only as reliable as each call site's own try/finally — which a crash, OOM, or timeout kill bypasses entirely. Add scripts/node-test-tmpdir.ts: it wraps the whole `node --test` invocation as a child process, redirecting TMPDIR to one disposable, pid-tagged directory (shared root/prefix with the Vitest lane) and removing it from the process 'exit' event, which fires on normal completion, a thrown error, or a forwarded SIGINT/SIGTERM alike. Every node --test script now runs through it. check-tmpdir-leaks.ts already scans by root/prefix, so it covers both mechanisms with no changes to its detection logic. Verified: a node --test process that mkdtemp's then gets SIGKILL'd leaves a directory behind unwrapped; wrapped and SIGTERM'd, TMPDIR is redirected and the directory is gone with no orphaned processes. All 13 wrapped lanes and the full Vitest suite (5,591 tests) pass with zero residual agent-device-test-run-* directories after the run. Fixes #1595 * test(tmpdir): ratchet every node --test script through the wrapper The 13 lanes wrapped in package.json were a one-time hand sweep with nothing enforcing the pattern going forward — a 14th node --test script added later without scripts/node-test-tmpdir.ts would silently reopen #1595 for that one lane. Add a structural check to scripts/node-test-tmpdir.test.ts (now part of check:tmpdir-leaks:test) that reads package.json and fails if any script invokes `node ... --test` without routing through the wrapper. Dumb string matching over the scripts map, no shell parsing, with an explicit (currently empty) NODE_TEST_WRAPPER_BYPASS_ALLOWLIST for any lane that must legitimately bypass it. Verified it both passes on the current package.json and fails when a synthetic unwrapped `node --test` script is added. * fix(test): preserve the Swift cache and close the raw node --test bypasses Review on #1661 found two gaps: 1. The wrapper only overrode TMPDIR, so it discarded and forced a recompile of the durable Swift compiler cache every run instead of mirroring vitest-tmpdir-global-setup.ts's carve-out for it. Read os.tmpdir() before the child's TMPDIR redirect takes effect and set AGENT_DEVICE_SWIFT_CACHE_DIR from that (only when unset), same as the Vitest lane — the two now share one durable cache instead of each discarding and recompiling their own. Added a probe assertion (scripts/node-test-tmpdir.test.ts) that fails without the fix and passes with it (verified both ways). 2. docs/agents/testing.md documented raw `node --test` commands for the iOS smoke files, and the android/ios/conformance-regenerate/nightly workflows invoked `node --test` directly outside package.json. Routed all of them through scripts/node-test-tmpdir.ts so the documented local commands and CI lanes get the same crash/timeout-safe cleanup the package.json scripts already have.
1 parent cc94340 commit ac52281

9 files changed

Lines changed: 336 additions & 28 deletions

File tree

‎.github/workflows/android.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ jobs:
5252
run: echo "Android fixture APK source=${{ steps.fixture-app.outputs.source }}"
5353

5454
- name: Run Android emulator catalog coverage contract
55-
run: node --test test/integration/smoke-android-emulator-coverage.test.ts
55+
run: node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-android-emulator-coverage.test.ts
5656

5757
- name: Run Android smoke checks
5858
uses: reactivecircus/android-emulator-runner@b530d96654c385303d652368551fb075bc2f0b6b # v2.35.0
@@ -68,7 +68,7 @@ jobs:
6868
test -n "$ANDROID_SERIAL"
6969
pnpm build
7070
pnpm clean:daemon
71-
AGENT_DEVICE_ANDROID_E2E=1 AGENT_DEVICE_ANDROID_E2E_TIER=smoke AGENT_DEVICE_ANDROID_SERIAL="$ANDROID_SERIAL" AGENT_DEVICE_FIXTURE_APP_PATH="${{ steps.fixture-app.outputs.apk-path }}" AGENT_DEVICE_FIXTURE_APP_ID="${{ steps.fixture-app.outputs.app-id }}" node --test test/integration/smoke-android-emulator.test.ts
71+
AGENT_DEVICE_ANDROID_E2E=1 AGENT_DEVICE_ANDROID_E2E_TIER=smoke AGENT_DEVICE_ANDROID_SERIAL="$ANDROID_SERIAL" AGENT_DEVICE_FIXTURE_APP_PATH="${{ steps.fixture-app.outputs.apk-path }}" AGENT_DEVICE_FIXTURE_APP_ID="${{ steps.fixture-app.outputs.app-id }}" node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-android-emulator.test.ts
7272
node --experimental-strip-types src/bin.ts test test/integration/replays/android/01-settings.ad --retries 2 --report-junit test/artifacts/replays-android-smoke.junit.xml
7373
7474
- name: Upload Android artifacts

‎.github/workflows/conformance-regenerate.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,4 +63,4 @@ jobs:
6363
# The seal check also runs per-PR, but assert it here too so a regeneration
6464
# that produced an unsealed/mis-sealed fixture cannot pass silently.
6565
- name: Verify fixture seals and conformance
66-
run: node --experimental-strip-types --test packages/maestro/test/conformance/verify.test.ts
66+
run: node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test packages/maestro/test/conformance/verify.test.ts

‎.github/workflows/ios.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ jobs:
147147
AGENT_DEVICE_IOS_UDID: ${{ steps.ios-simulator.outputs.simulator-udid }}
148148
run: |
149149
pnpm build
150-
node --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
150+
node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
151151
152152
- name: Assert simulator automation preserved host focus
153153
if: ${{ always() }}

‎.github/workflows/replays-nightly.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ jobs:
142142
node --experimental-strip-types src/bin.ts replay examples/test-app/replays/drag-android.ad --platform android --serial "$ANDROID_SERIAL" --session nightly-android-target-drag --json
143143
pnpm clean:daemon
144144
FULL_STARTED_AT="$(date +%s)"
145-
AGENT_DEVICE_ANDROID_E2E=1 AGENT_DEVICE_ANDROID_E2E_TIER=full AGENT_DEVICE_ANDROID_SERIAL="$ANDROID_SERIAL" AGENT_DEVICE_FIXTURE_APP_PATH="${{ steps.fixture-app.outputs.apk-path }}" AGENT_DEVICE_FIXTURE_APP_ID="${{ steps.fixture-app.outputs.app-id }}" node --test test/integration/smoke-android-emulator.test.ts
145+
AGENT_DEVICE_ANDROID_E2E=1 AGENT_DEVICE_ANDROID_E2E_TIER=full AGENT_DEVICE_ANDROID_SERIAL="$ANDROID_SERIAL" AGENT_DEVICE_FIXTURE_APP_PATH="${{ steps.fixture-app.outputs.apk-path }}" AGENT_DEVICE_FIXTURE_APP_ID="${{ steps.fixture-app.outputs.app-id }}" node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-android-emulator.test.ts
146146
FULL_FINISHED_AT="$(date +%s)"
147147
echo "Android setup before emulator: $((BOOT_FINISHED_AT - ${{ steps.android-setup.outputs.seconds }}))s"
148148
echo "Android full scenario wall time: $((FULL_FINISHED_AT - FULL_STARTED_AT))s"
@@ -229,7 +229,7 @@ jobs:
229229
AGENT_DEVICE_IOS_UDID: ${{ steps.ios-simulator.outputs.simulator-udid }}
230230
run: |
231231
pnpm build
232-
node --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
232+
node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
233233
234234
- name: Run iOS physical device replay suite
235235
if: env.IOS_UDID != ''

‎docs/agents/testing.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -402,7 +402,7 @@ macOS focus.
402402
Run the static contract and documented live skip locally:
403403

404404
```bash
405-
node --test test/integration/smoke-ios-simulator-coverage.test.ts
405+
node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-ios-simulator-coverage.test.ts
406406
```
407407

408408
Run a live tier after booting a simulator and obtaining a current Release `.app`:
@@ -416,7 +416,7 @@ AGENT_DEVICE_IOS_UDID=<simulator-udid> \
416416
AGENT_DEVICE_FIXTURE_APP_PATH=<fixture.app> \
417417
AGENT_DEVICE_FIXTURE_APP_ID=com.callstack.agentdevicelab \
418418
AGENT_DEVICE_IOS_APP_EVENT_URL_TEMPLATE='agent-device-test-app:///automation?event={event}&payload={payload}' \
419-
node --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
419+
node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
420420
```
421421

422422
Use `AGENT_DEVICE_IOS_E2E_TIER=full` for the nightly subset. Step history, coverage reports,

‎package.json‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@
109109
"release:publish": "pnpm release:prepare && npm publish --ignore-scripts .tmp/release/*.tgz",
110110
"ad": "node bin/agent-device.mjs",
111111
"bench:help-conformance": "node scripts/help-conformance-bench.mjs",
112-
"maestro:conformance": "node --experimental-strip-types --test packages/maestro/test/conformance/verify.test.ts packages/maestro/test/conformance/differential/run.test.ts packages/maestro/test/conformance/differential/invariants.test.ts",
112+
"maestro:conformance": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test packages/maestro/test/conformance/verify.test.ts packages/maestro/test/conformance/differential/run.test.ts packages/maestro/test/conformance/differential/invariants.test.ts",
113113
"maestro:conformance:regenerate": "node --experimental-strip-types scripts/maestro-conformance/regenerate.mjs",
114114
"maestro:conformance:differential": "node --experimental-strip-types packages/maestro/test/conformance/differential/run.ts",
115115
"size": "node scripts/size-report.mjs",
@@ -118,7 +118,7 @@
118118
"mutation:baseline": "node --experimental-strip-types scripts/mutation/run.ts --update",
119119
"mutation:check": "node --experimental-strip-types scripts/mutation/run.ts --no-run",
120120
"mutation:affected": "node --experimental-strip-types scripts/mutation/run.ts --affected",
121-
"mutation:test": "node --experimental-strip-types --test scripts/mutation/*.test.ts",
121+
"mutation:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/mutation/*.test.ts",
122122
"lint": "oxlint . --deny-warnings",
123123
"format": "node ./node_modules/oxfmt/bin/oxfmt --write",
124124
"format:check": "node ./node_modules/oxfmt/bin/oxfmt --check",
@@ -127,19 +127,19 @@
127127
"fallow:baseline": "fallow dead-code --save-baseline fallow-baselines/dead-code.json --summary && fallow health --report-only --save-baseline fallow-baselines/health.json --summary",
128128
"check:fallow": "fallow audit",
129129
"check:affected": "node --experimental-strip-types scripts/check-affected/run.ts",
130-
"check:affected:test": "node --experimental-strip-types --test scripts/check-affected/model.test.ts scripts/check-affected/run.test.ts",
130+
"check:affected:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/check-affected/model.test.ts scripts/check-affected/run.test.ts",
131131
"check:coverage-changed": "node --experimental-strip-types scripts/coverage-changed/run.ts",
132-
"check:coverage-changed:test": "node --experimental-strip-types --test scripts/coverage-changed/model.test.ts scripts/coverage-changed/run.test.ts",
133-
"check:layering": "node --experimental-strip-types --test scripts/layering/model.test.ts scripts/layering/zone-policy.test.ts scripts/layering/daemon-modularity.test.ts scripts/layering/package-boundaries.test.ts scripts/layering/facade-exports.test.ts scripts/layering/bin-alias-fast-path.test.ts && node --experimental-strip-types scripts/layering/check.ts",
132+
"check:coverage-changed:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/coverage-changed/model.test.ts scripts/coverage-changed/run.test.ts",
133+
"check:layering": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/layering/model.test.ts scripts/layering/zone-policy.test.ts scripts/layering/daemon-modularity.test.ts scripts/layering/package-boundaries.test.ts scripts/layering/facade-exports.test.ts scripts/layering/bin-alias-fast-path.test.ts && node --experimental-strip-types scripts/layering/check.ts",
134134
"depgraph": "node --experimental-strip-types scripts/depgraph/build.ts",
135-
"depgraph:test": "node --experimental-strip-types --test scripts/depgraph/model.test.ts scripts/depgraph/affected.test.ts",
135+
"depgraph:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/depgraph/model.test.ts scripts/depgraph/affected.test.ts",
136136
"check:production-exports": "fallow dead-code --config fallow-production-exports.json --production --unused-exports --fail-on-issues",
137137
"check:bundle-owner-files": "node --experimental-strip-types scripts/check-bundle-owner-files.ts",
138138
"check:package": "node --experimental-strip-types scripts/check-package.ts",
139139
"check:command-docs": "vitest run --project unit-core src/__tests__/command-doc-coverage.test.ts",
140140
"check:replay-compat": "node --experimental-strip-types scripts/check-replay-compat-provenance.ts",
141141
"check:tmpdir-leaks": "node --experimental-strip-types scripts/check-tmpdir-leaks.ts",
142-
"check:tmpdir-leaks:test": "node --experimental-strip-types --test scripts/check-tmpdir-leaks-model.test.ts scripts/vitest-tmpdir-global-setup.test.ts",
142+
"check:tmpdir-leaks:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/check-tmpdir-leaks-model.test.ts scripts/vitest-tmpdir-global-setup.test.ts scripts/node-test-tmpdir.test.ts",
143143
"check:freerange": "fr",
144144
"check:quick": "pnpm lint && pnpm typecheck",
145145
"sync:mcp-metadata": "node scripts/sync-mcp-metadata.mjs",
@@ -165,17 +165,17 @@
165165
"test:maestro-compat": "vitest run --project unit-core packages/maestro src/daemon/adapters/maestro src/compat/__tests__/replay-input.test.ts",
166166
"test:coverage": "vitest run --coverage",
167167
"test:coverage:ci": "node --experimental-strip-types scripts/lib/contention-retry-run.ts --coverage",
168-
"check:contention-retry": "node --experimental-strip-types --test scripts/lib/contention-retry-policy.test.ts",
168+
"check:contention-retry": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/lib/contention-retry-policy.test.ts",
169169
"test:integration:provider": "vitest run --project provider-integration",
170170
"test:integration:progress": "node --experimental-strip-types scripts/integration-progress.ts",
171171
"test:integration:progress:check": "node --experimental-strip-types scripts/integration-progress.ts --check",
172-
"test:fixture-cache": "node --test test/ci/trusted-fixture-artifact.test.mjs",
172+
"test:fixture-cache": "node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/ci/trusted-fixture-artifact.test.mjs",
173173
"test:output-economy": "vitest run --project output-economy",
174-
"test:smoke:web": "pnpm build && node --test test/integration/smoke-web-platform.test.ts",
175-
"test:smoke": "node --test test/integration/smoke-*.test.ts",
176-
"test:integration:node": "node --test test/integration/*.test.ts",
174+
"test:smoke:web": "pnpm build && node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-web-platform.test.ts",
175+
"test:smoke": "node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-*.test.ts",
176+
"test:integration:node": "node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/*.test.ts",
177177
"test:integration": "pnpm test:integration:node && pnpm test:integration:provider",
178-
"test:concurrency-torture": "node --test test/integration/nightly/concurrency-torture.test.ts",
178+
"test:concurrency-torture": "node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/nightly/concurrency-torture.test.ts",
179179
"test:replay:ios": "node --experimental-strip-types src/bin.ts test test/integration/replays/ios/simulator",
180180
"test:replay:ios-device": "node --experimental-strip-types src/bin.ts test test/integration/replays/ios/device",
181181
"test:replay:android": "node --experimental-strip-types src/bin.ts test test/integration/replays/android",

‎scripts/check-tmpdir-leaks.ts‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,15 @@
11
// Fails if any agent-device-test-run-* directory under TEST_RUN_TMP_ROOT is
2-
// abandoned — its owning process has exited without running its
3-
// globalTeardown (crash, OOM, timeout kill). Directories owned by a still-
4-
// running process are left alone: a concurrent `vitest run` in another
5-
// worktree keeps its own directory present until its own teardown, which is
6-
// not a leak. See check-tmpdir-leaks-model.ts for the liveness check.
2+
// abandoned — its owning process has exited without running its cleanup
3+
// (crash, OOM, timeout kill). Directories owned by a still-running process
4+
// are left alone: a concurrent `vitest run` (or node --test lane, wrapped by
5+
// scripts/node-test-tmpdir.ts) in another worktree keeps its own directory
6+
// present until its own teardown, which is not a leak. See
7+
// check-tmpdir-leaks-model.ts for the liveness check.
78
//
8-
// Only covers vitest runs. node --test lanes (test:smoke,
9-
// test:integration:node, ...) still use the real os.tmpdir() unredirected.
9+
// Covers both redirection mechanisms sharing this root/prefix: Vitest's
10+
// globalSetup/globalTeardown (scripts/vitest-tmpdir-global-setup.ts) and the
11+
// node --test wrapper (scripts/node-test-tmpdir.ts, #1595) that every
12+
// `node --test` package.json script now runs through.
1013

1114
import path from 'node:path';
1215
import { findLeakedRunDirectories } from './check-tmpdir-leaks-model.ts';

0 commit comments

Comments
 (0)