Skip to content

Commit ea9609f

Browse files
committed
docs: record plan r17 (the race check of the TUI tests)
Plan section 10.20, log entry B.12, the baselines (r17, section 3.4), the CHANGELOG, AGENTS.md and the README of the env spy now say what the race check found, what changed, and what it did not cover.
1 parent 6438b59 commit ea9609f

7 files changed

Lines changed: 438 additions & 26 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ AgentMessage[] → transform_context() → convert_to_llm() → LangChain BaseMe
5050

5151
## Development
5252

53-
Python monorepo (`pi-agent-core`, `pi-agent-harness`, `pi-agent-cli`) + Rust TUI workspace `tui/` (vendored grok-build fork, binary `zypi`). Packages installed editable; tested via `pytest`. TUI: `cd tui && cargo check -p pi-pager-bin` (WSL; Cargo reads `CARGO_TARGET_DIR` from WSL env exclusively). Rust tests run through `python scripts/tui_baseline.py test` (manifest `scripts/tui_baseline.toml`, numbers in `docs/baselines/tui.md`); it clears `PI_HOME` and friends, which a WSL login shell may export pointing at the real home — do not run test binaries or `zypi` without clearing them. The run does not isolate `HOME` (plan D.9): some tests write into the real `~/.pi-python` and `/tmp`, so check both after a local run. Build on the Linux file system, not `/mnt/<drive>`. With a built `zypi`, `scripts/tui_pty/` drives it under a pseudo-terminal on Linux (smoke test, exit matrix, `-p` shutdown, the matrix of flags that resume a session, a probe of which permission modes the agent honours, a network audit (it shows that `zypi` itself opens no connection to a non-local address during a turn), a probe of the status line; README there). Two tools read the TUI tree: `python scripts/tui_callgraph.py keep --out docs/baselines/tui-support.md` regenerates the list of what the support crates still use (it reads source text only; the list is an estimate, `cargo check` has the last word), and `python scripts/tui_notices.py check` (needs cargo) says whether `tui/THIRD-PARTY-NOTICES` still matches `Cargo.lock`; run it after a dependency change, then `update --write`. The pager's ACP client is also run against the real Python agent without a terminal: `tui/crates/codegen/pi-pager/tests/python_agent_e2e.rs`, which needs `PI_E2E_PYTHON` (a Python with `pi_agent_cli` installed; TUI CI sets it and `PI_E2E_REQUIRED=1`, locally an unset variable makes the tests skip with a note). The tests of one Rust test binary run as threads of one process (CI: `cargo test`, 4 threads, a fixed order), and they share the environment variables and every `static`; the vendored tests were written for `cargo nextest` (one process per test), so their locks are partial. A test that sets such state can break another test in some runs only. `scripts/tui_envspy/` finds these tests (an `LD_PRELOAD` spy for environment variables and a shuffled many-thread stress script; README there). Every test that writes or reads one variable must hold the one lock that the other tests of that variable hold.
53+
Python monorepo (`pi-agent-core`, `pi-agent-harness`, `pi-agent-cli`) + Rust TUI workspace `tui/` (vendored grok-build fork, binary `zypi`). Packages installed editable; tested via `pytest`. TUI: `cd tui && cargo check -p pi-pager-bin` (WSL; Cargo reads `CARGO_TARGET_DIR` from WSL env exclusively). Rust tests run through `python scripts/tui_baseline.py test` (manifest `scripts/tui_baseline.toml`, numbers in `docs/baselines/tui.md`); it clears `PI_HOME` and friends, which a WSL login shell may export pointing at the real home — do not run test binaries or `zypi` without clearing them. The run does not isolate `HOME` (plan D.9): some tests write into the real `~/.pi-python` and `/tmp`, so check both after a local run. Build on the Linux file system, not `/mnt/<drive>`. With a built `zypi`, `scripts/tui_pty/` drives it under a pseudo-terminal on Linux (smoke test, exit matrix, `-p` shutdown, the matrix of flags that resume a session, a probe of which permission modes the agent honours, a network audit (it shows that `zypi` itself opens no connection to a non-local address during a turn), a probe of the status line; README there). Two tools read the TUI tree: `python scripts/tui_callgraph.py keep --out docs/baselines/tui-support.md` regenerates the list of what the support crates still use (it reads source text only; the list is an estimate, `cargo check` has the last word), and `python scripts/tui_notices.py check` (needs cargo) says whether `tui/THIRD-PARTY-NOTICES` still matches `Cargo.lock`; run it after a dependency change, then `update --write`. The pager's ACP client is also run against the real Python agent without a terminal: `tui/crates/codegen/pi-pager/tests/python_agent_e2e.rs`, which needs `PI_E2E_PYTHON` (a Python with `pi_agent_cli` installed; TUI CI sets it and `PI_E2E_REQUIRED=1`, locally an unset variable makes the tests skip with a note). The tests of one Rust test binary run as threads of one process (CI: `cargo test`, 4 threads, a fixed order), and they share the environment variables and every `static`; the vendored tests were written for `cargo nextest` (one process per test), so their locks are partial. A test that sets such state can break another test in some runs only. `scripts/tui_envspy/` finds these tests (an `LD_PRELOAD` spy for environment variables and a shuffled many-thread stress script; README there). Every test that writes or reads one variable must hold the one lock that the other tests of that variable hold. A test that changes a process-wide `static` that other tests read without a lock (the theme, the colour level, the minimal-mode flag, the bidi latch) runs its body in a process of its own: `crate::test_util::in_own_process(|| { … })` in `pi-pager`.
5454

5555
### Virtual environments (uv)
5656

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
12+
- **Tools that find tests which share process-wide state, and a helper that keeps such a test apart (TUI tests)**: `scripts/tui_envspy/` and `in_own_process`. All tests of one Rust test binary run as threads of one process, so they share the environment variables, the working directory and every `static`. The vendored tests were written for `cargo nextest` (one process per test), so their locks are partial, and a test that changes such state can break another test in some runs only. `envspy.sh` runs `cargo test` with an `LD_PRELOAD` shim over `getenv`, `setenv`, `unsetenv`, `putenv` and `chdir`, and `analyze.py` lists the tests that change a variable while another test runs, leave it changed, or read what another test has changed. `stress.sh` runs the lib tests of one package many times, each time in a random order (`--shuffle-seed`, so a failing run can be repeated) and with many test threads. It can also run a few named tests (a suspect pair, or a victim beside all tests of one module), and with `ENVSPY_TAG` it runs the tests through the spy. Linux only. 10 new `pytest` tests cover the analyzer. `crate::test_util::in_own_process(|| { … })` in `pi-pager` runs the body of a test in a child process (the same test binary, `--exact`, one test thread), and the test fails when the child fails; 57 tests use it, directly or through `with_rtl_bidi`, `with_theme_test_env` and `with_test_env`, and 2 new tests check it. The README of `scripts/tui_envspy/` has the rules for a test that changes process-wide state, and what the tools do not find. Numbers: section 3.4 of `docs/baselines/tui.md`.
13+
1014
### Changed
1115

1216
- **`zypi --continue`, `--resume` and `--session-id` ask the agent which session is meant (TUI)**: `zypi` used to look for the session in the session files of the old Rust agent. The Python agent writes another layout, so `--resume <title>` found no session, an unknown id failed only after the UI had started, a used `--session-id` left the UI open with `Session creation failed: Invalid params`, and `--continue` took the session that was created last (the first of the **Known issues** of 0.5.0). Now `zypi` starts the agent first and then asks it over ACP, with `session/list` (page after page) and `session/load`. It reads no session file for this. A session that does not exist ends the run before the event loop starts: the terminal is restored, the agent is stopped, the message is printed, and the exit status is 1. **Behaviour changes**: `--continue` and `--resume` without a value take the session of this directory with the latest activity (the time of its last write), not the one that was created last. `--resume <id>` finds the id in every directory and loads the session in the directory it belongs to. `--resume <title>` matches the title, which is the first message of the session, among the sessions of this directory, without regard to letter case and to spaces at the ends; two sessions with the same title are an error that lists their ids and last activity. `--session-id <id>` reads every page of the session list and refuses an id that any session has (`Session ID <id> is already in use.`, exit status 1); that costs one pass through the list (`scripts/session_list_bench.py` prints it as `walk all`), and an agent that offers no `session/list` skips the check and refuses a used id itself. The lookup stops after 30 s, the limit of the connect (`GROK_CONNECT_UI_TIMEOUT_SECS`). A session in which nobody typed has no file now, so `--continue` cannot take it (see the next entry). **Not solved, and left as it is**: the agent lists its session directory again for each page, so reading all pages costs pages × directory size, which grows faster than the number of sessions. `scripts/session_list_bench.py` (median of 3, Linux / Windows): `--session-id` takes 1.3 s / 1.8 s at 1,000 sessions and 28 s / 34 s at 5,000, which is over the 30 s limit on Windows; `--continue` takes 0.6 s / 0.8 s at 1,000 and 10 s / 12 s at 5,000. The owner decided that this is fine below 1,000 sessions (plan decision 18). Tests: 25 unit tests of the lookup against a scripted agent, 4 of the time limit and of a cancel, 3 end-to-end tests against the real agent, and `scripts/tui_pty/resume_matrix.py --strict` (13 of 13 pass; cases 5 and 11 were known gaps, and case 8 now needs exit status 1).
@@ -19,6 +23,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1923
- **The session store of `pi-shell`, and the code that only it used (TUI)**: `pi-shell` still held the code that wrote and read the session files of the old Rust agent (`session::storage` and `session::persistence`). Since the change above, no code in `zypi` calls it. It is deleted. The code that only it used goes with it: the session signals, the plan-mode snapshot, the announcement state, the state types of goals and workflows, the wire tags for replay, the image checks of the load path, `fork_filter_chat`, and the tests of all of these. The compiler found this second group: after the first deletion it gave 41 `dead_code` warnings, and the ceiling is 0. Two helpers that clean a session title stay, because the pager uses them (`session::title`, with 8 unit tests). 14,166 lines go and 190 come, in 53 files under `tui/`. **No behaviour changes.** One test attribute changes (the consent test of `pi-shell` takes the default `serial` key, like the other tests that set `GROK_HOME`): two tests used different keys, so they could run at the same time, and the `session::worktree` test then failed with a SQLite I/O error. That race was already in `c74cc18`; after the deletion it showed in every full run of the suite. The tests of `pi-shell` go from 1,056 to 864, and the sum of `reference_passed` in `scripts/tui_baseline.toml` from 8,706 to 8,514. `[storage] cleanup_ttl_days` was left in that change (nothing read it any more); the next entry takes it out. `gates.session_disk_reads` stays, as a guard against writing the store back.
2024
- **More dead code, ten unused dependencies, and the `[storage]` setting of `pi-shell` (TUI)**: the deletion above left more behind. The goal prompt templates and the research script `deep_research.rhai` are gone (15 files): no code has loaded them since the Rust agent runtime was removed in 0.5.0, and the compiler cannot see a file that no `include_str!` names. The modules `session::helpers`, `session::worktree` and `util::config::worktree` are gone: nothing in `zypi` reaches them. The `[storage]` section of the configuration is gone: `cleanup_ttl_days` cleaned the old session files, and nothing reads it. A `config.toml` that still has a `[storage]` section loads as before, and the section is reported as an unrecognized key, like any other unknown section. Ten dependencies of `pi-shell` that no source file names are removed: `urlencoding`, `base64`, `pi-fast-worktree`, `pi-compaction-transcript`, `pi-workflow`, `tdigests`, `nix`, and the test dependencies `pretty_assertions`, `pi-test-utils` and `tokio-util`. The dependency graph of `zypi` goes from 980 to 970 packages on Linux, and `THIRD-PARTY-NOTICES` loses five entries (`rhai`, `rhai_codegen`, `smartstring`, `tdigests`, `thin-vec`). 1,644 lines go and 13 come, in 28 files under `tui/`. **No behaviour changes.** The tests of `pi-shell` go from 864 to 838 (27 tests go with the deleted code, and one is new). The ceilings in `scripts/tui_baseline.toml` go down with the code: `leader` 245 to 243, `x.ai` 304 to 297, the dependency graph 980 to 970, and the sum of `reference_passed` 8,514 to 8,488. `pi-workflow` is still a member of the workspace, but no other crate depends on it now.
2125

26+
### Fixed
27+
28+
- **TUI tests that failed in some runs, and five that left environment variables changed**: stress runs (random order, 32 test threads) found 19 tests of `pi-shell` and `pi-pager` that fail in some runs, and 5 tests that leave environment variables changed. All are fixed in the tests. **No product code changes.** Four causes were defects of the tests. Two locks that do not keep each other out protected `GROK_SUBAGENTS` (`config::tests::subagents_config_models_with_env_var_enables`); the locks are now shared. A test that does not hold the lock of the process-wide trace sink wrote a line into it (`memory_trace::tests::sample_events_are_valid_jsonl_and_rotate`); the test now checks the sample lines only. A child process that another test forks keeps a copy of the lock's file descriptor until it calls `exec`, so the lock was still busy after the guard dropped (`auth::manager::lock::tests::dropping_the_guard_silences_the_heartbeat_before_anyone_else_can_hold_the_lock`); the test now retries for up to 5 s (a fork-storm experiment: 363 of 1,500 rounds busy with a forking test beside it, 0 of 1,500 without). Four `persist_permission_mode_*` tests and `trust_folder_grants_and_resolves` set `GROK_HOME` (the last one `GROK_TEST_VERSION` too) and did not put the old value back, so the later tests of the process read a deleted directory or saw a simulated release build (the spy counted about 12,900 such reads in one run); they put the values back now. The other 16 tests failed because of process-wide `static`s: the colour level and the native palette, the minimal-mode flag, the bidi latch, the embedded-modal flag, the voice gate and the theme. Hundreds of tests read these without a lock, so a lock cannot keep a writer away from them. The 57 tests that change them now run in a process of their own (`in_own_process`). Failed runs of the lib tests of `pi-pager` (random order, 32 test threads): 6 of 32 before, 0 of 60 after, and 0 of 32 with new seeds; `pi-shell`: 2 of 20 before, 0 of 20 after. **Not explained**: `blank_quote_line_survives_drag_copy_as_blank` failed once and `command_palette_search_bar_cursor_only_when_focused` failed in 1 of 60 runs; neither failed again after the change (the second one probably reads the vim-mode setting that 15 tests write). About 140 tests write other settings of the appearance cache (`set_show_thinking_blocks`, `set_group_tool_verbs` and others). They are not isolated, and no test is known to fail because of them. The sum of `reference_passed` is 8,490 (it was 8,488; the two new tests check `in_own_process`).
29+
2230
## [0.5.0] - 2026-10-13
2331

2432
`zypi` no longer has an agent of its own. It draws the terminal screen and talks ACP to the Python agent (`python -m pi_agent_cli`), and that agent is now the only agent. **Removed** says what went with the Rust agent runtime. **Known issues** says what does not work yet. This release also has the Phase 6 and Phase 7 changes to the Python packages: project trust, a sandbox for workflow scripts, DeepSeek thinking with tools, and tool annotations.

0 commit comments

Comments
 (0)