Skip to content

Commit 0c89d13

Browse files
authored
fix(tests): guard TMPDIR in CI and close the last test leaks (#1672)
* test(computer-use): accept a readback timeout in the normalization mismatch test Port of standalone computer-use-linux #431 (968aee5). The last poll of the editable text readback can start just before the one-second deadline and still be reading when it passes, which reports ReadbackTimedOut instead of MismatchAfterDeadline. The two other mismatch tests already accept both outcomes, since both leave the write unverified and neither allows a replay. The embedded src/atspi_tree.rs now equals standalone's at 968aee5. The crate stays 0.7.47. * ci: run the Node tests through the TMPDIR guard CI's Node step ran its own list of test files and never used the guard in scripts/ci/run-node-checks.sh, and the guard's glob missed three test.js files CI ran: the upstream package watchdog test and the quit-confirmation and shell-env core patch tests. The Node step now runs run-node-checks.sh all: a syntax check of every tracked .js file, then every *.test.js and test.js in a fresh TMPDIR, failing when a run leaves anything there. The glob takes every test.js, so CI runs exactly what the guard checks, including the new socket helper's unit test and suites the old list skipped. github-actions-workflows.test.js checks that the Node step is the guarded runner and that the runner's glob finds every tracked test file. * fix(appshots): check XInput 2 before the bare modifier stream starts The monitor started xinput test-xi2 and, 0.1 s later, printed permission-denied if the stream had already exited, else ready. test-xi2 fails at once when the X server is unreachable or has no XInput 2, but under load that exit can come after 0.1 s, and the monitor then printed ready and stopped. The monitor now runs xinput --version first and waits for it. That command opens the display and reads the server's XInput version, the same preconditions, so the answer no longer depends on timing. A missing or pre-2 server version gives permission-denied. The 0.1 s check stays for a stream that dies after a good preflight. Tests: a slow xinput that cannot reach the server and a server with XInput 1.5 must give permission-denied; the old monitor printed ready for both. A stream that dies after the preflight still leaves no stdbuf directory, whichever way the short check goes. * fix(global-dictation): stop the watcher's subshells before killing their children The release watcher's cleanup ran pkill -TERM -P on each helper subshell and then killed the subshell. A subshell that forked between the two, such as the watchdog before its sleep or the parent check between two sleep 1, left that child running after the watcher, up to the watchdog's 1800 s. A reviewer saw one such sleep once under heavy load. Cleanup now sends SIGSTOP to every subshell first. A stopped process cannot fork, so pkill -P sees every child it will ever have; the SIGTERM then waits until SIGCONT. A fork already in progress when the stop arrives completes before the stop takes effect, so its child is listed too. The race did not reproduce with the real watcher in 1200 parallel runs. A model that copies the watcher's cleanup() verbatim, with one subshell that forks 0.3 s late and a pkill that takes 0.5 s, leaves the late child running in 10 of 10 runs with the old cleanup and in 0 of 10 with this one. * test: keep the last test fixtures in TMPDIR and inside their try - Six shared-app-server-socket tests bound their socket before the try that removes the fixture, so a failed bind left it in TMPDIR. With listenUnix made to fail, the old file leaves 6 fixtures and this one none. - The GNOME dictation tests kept their scratch under ~/.cache/codex-desktop-dev, outside TMPDIR, so the guard could not see a leftover. They now use os.tmpdir(). The private-bus test's scratch holds a dbus-daemon socket, so it is placed with makeSocketTempDir under dbus-daemon's 99-byte limit; it passes with a 48 and a 92-byte TMPDIR. * docs(changelog): describe the final test and monitor fixes * fix: address review of the final TMPDIR and monitor fixes - Release watcher: cleanup ignores INT, TERM and HUP while it runs, and stops, signals and continues each helper subshell in one pass. A second signal during cleanup used to run the exit trap at once; on main that left the helpers running, and with the first fix it left them stopped. With a slowed pkill and a second SIGTERM 0.4 s after the first, main leaves 3 processes, the first fix 4 (three of them stopped), and this none. - A regression test for the cleanup race uses the real watcher: its parent check's sleep ignores SIGTERM and lasts 0.4 s, and pkill takes 1 s, so the loop forks again after pkill -P has listed the children. On main the late sleep outlives the watcher in 3 of 3 runs; here it never does. - shared-app-server-socket: a seventh test that bound before its try now binds inside it. - AppShots monitor: the xinput --version preflight runs under timeout 5 when timeout exists, so a server that never answers ends in permission-denied instead of silence. The startup test is renamed for what it tests, and the post-preflight test checks the exit status. - The workflow test finds the runner's test glob by its own line. - The changelog says run-node-checks.sh all also syntax-checks every .js file, and covers the second-signal fix. * fix: address the final review of the monitors and fixtures - Release watcher: cleanup ends each stopped subshell with SIGKILL instead of SIGTERM and SIGCONT. bash blocks SIGTERM from just before a fork until it returns, so a subshell stopped in that window resumed on SIGCONT, finished the fork and only then died, leaving the new child. With a shim that delays fork() in the helpers, main and the SIGTERM/SIGCONT cleanup left the watchdog's sleep and a sleep 1 in 3 of 3 runs; this leaves nothing. The subshells hold no state to clean up. - The race test now reports a parent-check sleep that outlived its helper instead of looking for it after the watcher exits, so it fails on main in 3 of 3 runs. It and a new test for a second SIGTERM during cleanup find a run's processes by the run's TMPDIR in their environment, and kill only those. The second-signal test fails on main in 3 of 3 runs. - AppShots monitor: the xinput --version preflight has no time limit again. An Xwayland started on demand can take longer than any fixed limit, and the stream itself waited for the server before. - package-common's permission tests keep their fixture under TMPDIR, where the guard sees it, instead of ~/.cache/codex-desktop-dev. - shared-app-server-socket: the stale-inode test closes its server in its finally block too. - The GNOME private-bus test comments no longer say the bus is on disk. - AGENTS.md and CONTRIBUTING.md name `bash scripts/ci/run-node-checks.sh all` among the validation commands. The changelog links agent-sh/computer-use-linux#431 and describes the SIGKILL cleanup. * fix(appshots): end a waiting preflight with the monitor The xinput --version preflight ran before the monitor installed its traps and in a command substitution, so while an X server did not answer, SIGTERM killed the monitor and left xinput running, and the monitor never noticed that its parent had exited. The traps and cleanup now come first, and the preflight runs in a process substitution that the monitor reads with a one-second timeout, checking its parent each time. Cleanup kills the preflight. It still has no time limit of its own. A coprocess would not do: bash drops a coprocess's descriptor once it exits, and a fast answer can exit before the read. Two tests cover a preflight that never answers: one sends SIGTERM, one lets the parent exit. Both fail on the previous commit and pass here. The release watcher tests also find processes whose TMPDIR is a directory under the run's, such as the stream's stdbuf directory, so a failing run cannot leave those behind. * fix(appshots): treat a stream gone before its first read as dead Bash unsets a coprocess's pid and descriptor variables once it exits. A stream that fails at once can exit before the monitor reads them, and set -u then stopped the monitor with an unbound-variable error and no output. The monitor now reads both with empty defaults and reports permission-denied when either is gone. With the monitor held 100 ms after each fork, the previous commit failed this way in 3 of 3 runs, as main does; this commit reports permission-denied in 3 of 3. The watcher test stubs exec their final sleep 10, so the watcher's kill of a stub ends that sleep too. The changelog says SIGTERM instead of "a stop signal". * fix(ci): run Node tests without --test-force-exit --test-force-exit can drop finished tests from the report and still exit 0 (nodejs/node#64833, open; its fix PRs nodejs/node#64857 and #64875 are unmerged). With the flag, 8 of 11 local full-suite runs on Node 26.12 and 24.21 reported up to 47 fewer tests than ran; without it, 11 of 11 reported every test in the same time. Each of the 96 test files exits on its own when run alone without the flag, and the timeout watchdog still bounds a leaked handle. * test: end the processes the bridge and Dock lock tests start The host-service tests removed their fixture while the detached bridge was still shutting down: one bridge outlived the file in 3 of 3 runs. They now take the socket path before the launch, await the owner's unregister, and wait up to 3 s for the bridge named in the lock to exit (checking its start time), then kill it; 0 of 3 after. The Dock icon lock test killed only flock, whose child keeps the lock and its sleep, so the file waited about 2.1 s for it after the test ended. The holder now reads its stdin and keeps the lock until the test closes it or dies: the wait is 0.15 s, node's own exit, and a SIGKILLed run leaves no holder 1 s later (3 of 3). The CHANGELOG folds the --test-force-exit removal into the Node test entry, since no release shipped the flag, and states when the watchdog applies.
1 parent 9997658 commit 0c89d13

17 files changed

Lines changed: 652 additions & 145 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -30,18 +30,9 @@ jobs:
3030
- name: Guided installer tests
3131
run: python3 scripts/feature-picker.test.py
3232
- name: Node tests
33-
run: |
34-
node --test scripts/ci/*.test.js
35-
node --test scripts/lib/upstream-linux-package.test.js \
36-
scripts/lib/install-verification.test.js
37-
node --test scripts/lib/package-common.test.js
38-
node --check scripts/ci/verify-official-linux-features.js
39-
node --test scripts/automation/upstream-linux-package-watchdog/test.js
40-
node --test scripts/patch-linux-window-ui.test.js scripts/patches/descriptor.test.js \
41-
scripts/patches/lib/global-shortcuts-portal.test.js \
42-
scripts/patches/lib/minified-js.test.js scripts/patches/lib/assets.test.js \
43-
scripts/patches/core/*/test.js \
44-
scripts/lib/linux-features.test.js linux-features/*/test.js
33+
# Syntax-checks every tracked .js file, then runs every *.test.js and
34+
# test.js in a fresh TMPDIR and fails when a run leaves anything there.
35+
run: bash scripts/ci/run-node-checks.sh all
4536
- name: Smoke tests
4637
run: bash tests/scripts_smoke.sh
4738

‎AGENTS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,7 @@ bash -n install.sh scripts/lib/*.sh launcher/start.sh.template
143143
bash tests/scripts_smoke.sh
144144
node --test scripts/lib/upstream-linux-package.test.js
145145
node --test scripts/patch-linux-window-ui.test.js scripts/lib/linux-features.test.js linux-features/*/test.js
146+
bash scripts/ci/run-node-checks.sh all # every Node test in a fresh TMPDIR, as CI runs them
146147
cargo test -p codex-update-manager
147148
cargo clippy -p codex-update-manager --all-targets -- -D warnings
148149
nix flake check

‎CHANGELOG.md‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -319,10 +319,40 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
319319
fixtures move to `XDG_RUNTIME_DIR` when a socket path would not fit under
320320
`TMPDIR`, so the suite passes with a long `TMPDIR` too; without
321321
`XDG_RUNTIME_DIR` they fail with the limit and the path.
322-
`scripts/ci/run-node-checks.sh`, a standalone script that runs the
323-
`*.test.js` files and the feature `test.js` files, now runs them with a
324-
fresh `TMPDIR` and fails, naming the entries, when a run leaves any. CI and
325-
`scripts/ci-local.sh` run their own lists of Node tests and do not use it.
322+
`scripts/ci/run-node-checks.sh all` syntax-checks every tracked `.js` file,
323+
then runs every `*.test.js` and `test.js` with a fresh `TMPDIR` and fails,
324+
naming the entries, when a run leaves any. CI's Node step runs it, so CI
325+
runs exactly the files the guard checks; `scripts/ci-local.sh` still runs
326+
its own list. The GNOME dictation and package permission tests keep their
327+
scratch under `TMPDIR` too, and the shared app-server tests bind their
328+
sockets inside the `try` that removes them. The host-service tests wait for
329+
their detached bridge to exit and kill it if it still runs after 3 s. The
330+
Dock icon lock test's holder keeps the lock only while the test holds its
331+
stdin open. The runner does not pass `--test-force-exit`. That flag can drop
332+
finished tests from the report and still exit 0 (nodejs/node#64833). With
333+
it, 8 of 11 local runs on Node 24 and 26 reported up to 47 fewer tests than
334+
ran. A file that a leaked handle holds open fails the run with status 124
335+
when the runner's 300 s `timeout` watchdog ends it. CI requires `timeout`; a
336+
local run without it has no limit.
337+
- The AppShots bare modifier monitor waits for `xinput --version` before it
338+
starts its XInput 2 stream, and reports `permission-denied` when the X
339+
server cannot be reached or has no XInput 2. Before, it judged the stream
340+
by whether it still ran 0.1 s after the start, so under load a failing
341+
`xinput` could print `ready` and then stop. SIGTERM or the app's exit
342+
still ends the monitor and the check while it waits. A stream that exits
343+
before the monitor reads its coprocess also gives `permission-denied`;
344+
before, the monitor stopped with an unbound-variable error and no output.
345+
- The global dictation release watcher stops each helper subshell, signals
346+
its children, then ends the subshell with SIGKILL. A child that a subshell
347+
forked between `pkill -P` and its own end, such as the watchdog's `sleep`,
348+
no longer outlives the watcher. A second SIGINT, SIGTERM or SIGHUP during
349+
cleanup no longer cuts it short, which left the helpers running.
350+
- The embedded Computer Use crate's
351+
`editable_text_normalization_mismatch_is_not_verified` test accepts
352+
`ReadbackTimedOut` as well as `MismatchAfterDeadline`, like the two other
353+
mismatch tests (agent-sh/computer-use-linux#431). Its last poll can start
354+
just before the one-second deadline and still be reading when it passes.
355+
Both outcomes leave the write unverified, and neither allows a replay.
326356
- Desktop entries no longer offer a "New Window" action. It relied on the
327357
retired multi-instance launcher (`--new-instance`, `CODEX_MULTI_LAUNCH`),
328358
which the official app ignores, so it did nothing while the app was running

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ bash -n install.sh scripts/lib/*.sh launcher/start.sh.template
117117
bash tests/scripts_smoke.sh
118118
node --test scripts/lib/upstream-linux-package.test.js
119119
node --test scripts/patch-linux-window-ui.test.js scripts/lib/linux-features.test.js linux-features/*/test.js
120+
bash scripts/ci/run-node-checks.sh all # every Node test in a fresh TMPDIR, as CI runs them
120121
cargo test -p codex-update-manager
121122
cargo clippy -p codex-update-manager --all-targets -- -D warnings
122123
./scripts/ci-local.sh pr

‎computer-use-linux/src/atspi_tree.rs‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2225,7 +2225,16 @@ mod tests {
22252225
.unwrap(),
22262226
);
22272227

2228-
assert_eq!(outcome, EditableTextSetOutcome::MismatchAfterDeadline);
2228+
// A last poll can cross the same fixed deadline. Both outcomes mean
2229+
// an unverified write; neither permits verification or replay.
2230+
assert!(
2231+
matches!(
2232+
outcome,
2233+
EditableTextSetOutcome::MismatchAfterDeadline
2234+
| EditableTextSetOutcome::ReadbackTimedOut
2235+
),
2236+
"{outcome:?}"
2237+
);
22292238
let state = state.lock().unwrap();
22302239
assert_eq!(state.mutations, 1);
22312240
assert!(state.character_count_reads > 1);

‎linux-features/appshots/bin/bare-modifier-monitor‎

Lines changed: 83 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,81 @@ if [ -z "${DISPLAY:-}" ] || ! command -v xinput >/dev/null 2>&1; then
112112
exit 1
113113
fi
114114

115+
parent_pid="$PPID"
116+
monitor_pid=""
117+
preflight_pid=""
118+
event_fd=""
119+
stdbuf_dir=""
120+
cleanup_started=0
121+
122+
cleanup() {
123+
[ "$cleanup_started" -eq 0 ] || return
124+
cleanup_started=1
125+
126+
if [ -n "$preflight_pid" ]; then
127+
kill -KILL "$preflight_pid" >/dev/null 2>&1 || true
128+
wait "$preflight_pid" >/dev/null 2>&1 || true
129+
fi
130+
131+
if [ -n "$monitor_pid" ]; then
132+
kill "$monitor_pid" >/dev/null 2>&1 || true
133+
sleep 0.05
134+
kill -KILL "$monitor_pid" >/dev/null 2>&1 || true
135+
wait "$monitor_pid" >/dev/null 2>&1 || true
136+
fi
137+
138+
# Only the directory this script made.
139+
case "$stdbuf_dir" in
140+
*/codex-appshots-stdbuf.??????) rm -rf -- "$stdbuf_dir" 2>/dev/null || true ;;
141+
esac
142+
}
143+
trap cleanup EXIT
144+
trap 'exit 143' INT TERM
145+
146+
# xinput test-xi2 fails at once when the X server is unreachable or has no
147+
# XInput 2, but how soon that exit shows depends on load, so the check after
148+
# starting the stream below can miss it. xinput --version opens the display
149+
# and reads the server's XInput version, the same preconditions. It is awaited
150+
# for as long as the server takes, as the stream itself would wait for it (an
151+
# Xwayland started on demand can take a while), but it runs in the background
152+
# so that SIGTERM and the parent's exit still end the monitor, and cleanup
153+
# ends the probe with it. It is a process substitution, not a coprocess: bash
154+
# drops a coprocess's descriptor once it exits, and a fast answer can exit
155+
# before the read.
156+
exec {preflight_fd}< <(exec xinput --version 2>/dev/null)
157+
preflight_pid="$!"
158+
xinput_version_output=""
159+
partial=""
160+
while :; do
161+
line=""
162+
read_status=0
163+
IFS= read -r -t 1 -u "$preflight_fd" line || read_status=$?
164+
if [ "$read_status" -eq 0 ]; then
165+
xinput_version_output+="$partial$line"$'\n'
166+
partial=""
167+
elif [ "$read_status" -gt 128 ]; then
168+
# No full line yet. Keep what arrived, and wait on while the parent lives.
169+
partial+="$line"
170+
kill -0 "$parent_pid" >/dev/null 2>&1 || exit 1
171+
else
172+
xinput_version_output+="$partial$line"
173+
break
174+
fi
175+
done
176+
exec {preflight_fd}<&-
177+
preflight_status=0
178+
wait "$preflight_pid" || preflight_status=$?
179+
preflight_pid=""
180+
[ "$preflight_status" -eq 0 ] || xinput_version_output=""
181+
server_xi_version="${xinput_version_output##*XI version on server: }"
182+
case "$server_xi_version" in
183+
[2-9].*|[1-9][0-9]*.*) ;;
184+
*)
185+
echo "permission-denied"
186+
exit 1
187+
;;
188+
esac
189+
115190
xmodmap_output="$(xmodmap -pke 2>/dev/null || true)"
116191
IFS=$'\t' read -r key_symbols fallback_codes < <(select_keys "$key")
117192
key_codes="$(find_keycodes "$key_symbols" "$fallback_codes")"
@@ -143,31 +218,6 @@ last_tap_ms=0
143218
last_tap_code=""
144219
active=0
145220

146-
parent_pid="$PPID"
147-
monitor_pid=""
148-
event_fd=""
149-
stdbuf_dir=""
150-
cleanup_started=0
151-
152-
cleanup() {
153-
[ "$cleanup_started" -eq 0 ] || return
154-
cleanup_started=1
155-
156-
if [ -n "$monitor_pid" ]; then
157-
kill "$monitor_pid" >/dev/null 2>&1 || true
158-
sleep 0.05
159-
kill -KILL "$monitor_pid" >/dev/null 2>&1 || true
160-
wait "$monitor_pid" >/dev/null 2>&1 || true
161-
fi
162-
163-
# Only the directory this script made.
164-
case "$stdbuf_dir" in
165-
*/codex-appshots-stdbuf.??????) rm -rf -- "$stdbuf_dir" 2>/dev/null || true ;;
166-
esac
167-
}
168-
trap cleanup EXIT
169-
trap 'exit 143' INT TERM
170-
171221
# uutils stdbuf makes a temporary directory in TMPDIR and then execs its
172222
# command, so nothing removes it (uutils/coreutils#13939). Give it a directory
173223
# this script removes on exit. GNU stdbuf writes nothing there.
@@ -183,8 +233,14 @@ if command -v stdbuf >/dev/null 2>&1; then
183233
stdbuf_dir="$(mktemp -d "${TMPDIR:-/tmp}/codex-appshots-stdbuf.XXXXXX" 2>/dev/null)" || stdbuf_dir=""
184234
fi
185235
coproc APPSHOT_XINPUT { start_xinput; }
186-
monitor_pid="$APPSHOT_XINPUT_PID"
187-
event_fd="${APPSHOT_XINPUT[0]}"
236+
# Bash unsets these variables once the producer exits, which a stream that
237+
# fails at once can do before the next line runs. That stream already died.
238+
monitor_pid="${APPSHOT_XINPUT_PID:-}"
239+
event_fd="${APPSHOT_XINPUT[0]:-}"
240+
if [ -z "$monitor_pid" ] || [ -z "$event_fd" ]; then
241+
echo "permission-denied"
242+
exit 1
243+
fi
188244

189245
sleep 0.1
190246
if ! kill -0 "$monitor_pid" >/dev/null 2>&1; then

0 commit comments

Comments
 (0)