Repository navigation
refactor(tui): delete the session store of pi-shell (plan decision 10) - #26
Merged
Merged
Conversation
…on 17) A new session was a file from session/new on: a header and nothing else. A user who started the TUI and left, left such a file behind. session/list showed it as "(no messages)", and zypi --continue could take it for the latest session. session/new now makes a deferred session (DeferredJsonlSessionStorage, in the harness). It keeps its entries in memory until the first message entry. That entry writes the header, the entries held before it (a model choice, a name, a leaf move) and itself in one go. After that each entry is appended as before. The write comes before the entry is recorded, so a failed write leaves no trace of the message and the next message tries again. A session in which nobody typed has no file: nothing to list, load or continue. Sessions that older versions left with no message keep the title "(no messages)". An id the client chose (--session-id) must stay taken while its session has no file, so the agent keeps the ids of its open sessions in a set. The set leaves an id when the session is closed or deleted, or when the session could not be opened. Another process cannot see that set. So the first prompt of a session with a chosen id looks for the id in the files once more, and refuses it with the same error as session/new if a session has saved under it since. session/load and session/resume answer "not found" for a session that has no file: there is nothing to load.
…n 17) The agent now writes a session file with the first user message. Three checks that drive the real pieces had to change with it: - scripts/tui_pty/resume_matrix.py (zypi under a pseudo-terminal): the TUI makes a session at once, with no message, for `--session-id` and for `/new`. (A start with no flag makes none until the first message.) Two new cases leave such a session with no message and require that no session file exists. The first one comes first on purpose: under an agent that writes at once, its empty session is the newest one, so `--continue` and `--resume` with no argument take it and fail too. Two cases type a message and look for the file after it: `--session-id <fresh uuid>`, and `/new` and then a message. A case can have steps (`Step`). wait_for_run types the first one when the TUI is up (the model name on the border of the input box), and each next one when the screen shows that the one before took effect. The `/new` cases start with `--resume <id>`, so they do not depend on which session is the newest. scripts/tests/test_tui_pty.py tests the cases and wait_for_run with a fake terminal. Mutation check: with either of the two `repo.create` calls in agent.py made to write at once, the case for that call fails (and for the call that serves `--session-id`, the two cases that take the newest session fail too). - scripts/tui_pty/smoke.py: after a restart, the /resume picker used to list the new empty session first and the script pressed Down to skip it. That session has no file now and is not listed, so the earlier session is the first row and the script no longer presses Down. - python_agent_e2e.rs (the Rust ACP client against the real Python agent): the sessions that must be listed (51 for the paging test, the `existing` one for the id check) now get a first message. The id check also asserts what the agent does for a session that has no file yet: it refuses a second `session/new` with the same id (InvalidParams), and once the first message is sent the pager's own check finds the id through session/list. No test is added or removed on the Rust side (the count of python_agent_e2e stays 14).
A run of 15 one-line changes to the production code of this PR, each against the tests that watch it, found two that no test noticed: - The header of a deferred session took its time from a second look at the clock and not from the time in the file name. The test compared the two, but both looks fell in the same millisecond, so it passed by chance. It now gives every look at the clock a new time. - The second look at a chosen id (before the first message) also refused a session whose own file was there. No test sent a second message in a session with a chosen id. A new test sends three. After the change, both are caught. The other 13 were caught before.
…st message) and r14
- CHANGELOG.md, [Unreleased]: the agent writes the file of a session with
its first message. It lists the behaviour changes (no flag: no session
until the first message; `--session-id` and `/new`: a session at once,
with no file; a chosen id stays reserved while its session is open),
what library users get (`JsonlSessionRepo.create({"deferred": True})`),
and the tests. The entry for the cost of paging says it is left as it is.
- The embedded user guide (17-sessions.md), packages/pi-agent-cli/AGENTS.md
and docs/DESIGN.md section 6.1: a session in which nobody typed has no
file, so it is not listed and cannot be loaded.
- scripts/tui_pty/README.md (16 cases) and docs/baselines/tui.md.
- The plan (r14): the status, the answers to the four questions (#23 and
#24 merged, decision 18 not done, decision 17 done, decision 10 to do),
and the index of section 10.
- The execution log: section 10.17 (the merges, the design of decision 17,
what each ACP method does for a session without a file, the facts about
the TUI, the tests, two mutation runs and the two gaps they found, the
numbers, what is not verified), B.9 (how to repeat it) and the revision
record. One old line (about the empty first row of the /resume picker)
points to section 10.17.
Since ADR1 (PR 3) the pager reads no session file. The code in pi-shell that wrote and read those files has no caller left: session::storage (14 files) and session::persistence (12 files, 11 of them tests). Deleted with it, because the compiler then reported it as unused (41 dead_code warnings; the ceiling is 0): the session signals, the plan mode snapshot, the announcement state, the goal and workflow state types, the wire tags for replay, the image checks of the load path, fork_filter_chat, session::helpers::session_summary, and the test files that only tested these. Kept: the two helpers that clean a session title and that the pager uses on purpose (sanitize_and_cap_title, is_forbidden_title_char). They move to the new session::title, with 8 unit tests. GoalClassifierVerdict moves to extensions::notification, and ImageCompressionInfo stays in session::image_normalize. tui/: 52 files, +185 / -14,165 lines (38 files deleted, 1 added). The tests of pi-shell go from 1,056 to 864. No behaviour change. The [storage] cleanup_ttl_days setting is still accepted; nothing reads it now. tui_baseline.toml has the new numbers for pi-shell, and tui-support.md is generated again.
…ROK_HOME tests
`set_consent_answer_is_monotonic_per_account` set GROK_HOME under the named key
`serial(GROK_HOME)`. `EnvGuard` has no lock of its own, and every other test that
sets GROK_HOME uses the default `serial` key, so only the consent test could run at
the same time as the tests that read GROK_HOME without a guard
(`session::worktree::tests::resolve_worktree_by_id_or_path_*`).
When the two overlapped, the worktree test opened `worktrees.db` in the temporary
home of the consent test. The consent test then deleted that home at its end, and
the open connection failed with SQLite error 1802 ("disk I/O error", a failed stat
of the database file).
An LD_PRELOAD trace of one failing run showed it: the worktree test's thread opened
`<tmp>/worktrees.db`, a thread that ran the consent test removed the directory 130 ms
later, and no `fstat` call failed. Before the fix the failure was deterministic in
the full lib run with 12 test threads; at 4 threads it did not appear.
The fix is one attribute. No production code changes.
…ed) and r15 Decision 10, storage-layer part: the plan, the execution log (section 10.18, the pointer in appendix B.10, revision r15), the TUI baselines and the CHANGELOG now say that `session::storage` and `session::persistence` of `pi-shell` are gone, with the numbers (53 files under `tui/`, +190 / -14,166; `pi-shell` tests 1,056 -> 864; the sum of `reference_passed` 8,706 -> 8,514) and what was verified and what was not. Section 10.18 also holds the finding behind the test commit 332ba75: the consent test used the named serial key `GROK_HOME`, so it ran at the same time as the `session::worktree` tests and deleted the temporary home that their database lived in. The race is older than this PR (15 of 15 filtered runs fail at c74cc18). Plan D.9 now names one source of `worktrees.db` in the real home. `docs/baselines/tui-support.md` is regenerated (only the commit line changes). `scripts/tui_baseline.toml`: comment only (`pi-shell` 1,056 - 201 + 9 = 864).
…execution log The CI of 69932cf (it differs from 332ba75 by documents only) passed: `TUI CI` 8,514 passed / 0 failed / 21 ignored, `Gates + cargo check` green (graph 980, 0 warnings), `CI` (Ruff, pytest on Python 3.11, 3.12 and 3.13: 2,559 / 39 / 31). The execution log says what this does not show: the runner has 4 vCPU and runs 4 test threads, and the race between the consent test and `session::worktree` does not appear at 4 threads (before the fix as well). The proof of the fix is the 12-thread runs of section 10.18.
zy1233
added a commit
that referenced
this pull request
Oct 20, 2026
…used in pi-shell (plan decision 10) (#27) * refactor(tui): delete the goal prompt templates and the research script that nothing loads Nothing in the tree names these 15 files any more (git grep by file name: no hit). The last code that loaded them was removed with the Rust agent runtime in 455527c (#7); the files stayed, and rustc cannot see a file that no `include_str!` names, so no warning reported them. pi-shell/src/session/templates/ (14 Markdown prompt templates for the goal mode) and pi-shell/src/session/workflows/deep_research.rhai (584 lines). 15 files, 1,033 lines. No Rust code changes. * refactor(tui): delete the dead units session::worktree and session::helpers of pi-shell The call graph (scripts/tui_callgraph.py units) reports both as dead: nothing in the consumer build (pi-pager-bin, pi-pager-minimal, pi-update) reaches them. rustc does not warn, because what they hold is `pub(crate)` or a re-export. session::worktree (359 lines): a glob re-export of pi_workspace::worktree, two `From` impls between the shell's WorktreeType and the workspace's (nothing calls them), and 24 tests of types and functions that live in pi-workspace (serde shapes, build_candidate_list, BackgroundCopyContext, resolve_worktree_by_id_or_path). session::helpers (3 files, 61 lines): comments, and two tests of string shapes. The two `resolve_worktree_by_id_or_path` tests were the ones that opened worktrees.db in whatever GROK_HOME held; they are gone with the unit. pi-shell tests: 864 -> 838. pi_workspace::worktree has no user left in pi-shell after this; that is left for plan D.8. * refactor(tui): remove the [storage] section from the pi-shell config `[storage] cleanup_ttl_days` set how long old session files were kept. The code that read it went with the session store (#26), and the pager keeps no session file now. `StorageConfig` and the `storage` field of `Config` are deleted. A config file that still has the section loads. The section is reported as unrecognized, like any other unknown section: a new test pins that, and the "valid config" test no longer lists the section. `storage_mode` (the `--storage-mode` flag and `Config.storage_mode`) is a different setting. This change does not touch it. * refactor(tui): delete util::config::worktree of pi-shell `util::config::worktree` holds a `WorktreeType` enum (with `FromStr` and a `From` impl for `CreationMode`). Its one user was `session::worktree`, which the previous commit deletes. The baseline `check` shows it: the glob re-export of the module is an unused import (1 warning; the ceiling is 0). `pi-workspace` and `pi-workspace-types` have their own `WorktreeType` and do not use this one. The module has one test, `test_worktree_type_fromstr`; it goes with it. `pi-shell` has no other reference to `pi_fast_worktree` now, so that dependency goes in the next commit. * refactor(tui): remove the dependencies of pi-shell that no source file names Ten dependencies of `pi-shell` are not named in any source file of the crate outside comments (a text scan of `src/`, `tests/` and `build.rs`; the compiler has the last word): - `[dependencies]`: `urlencoding`, `base64`, `pi-fast-worktree`, `pi-compaction-transcript`, `pi-workflow`, `tdigests`, and `nix` (unix only). - `[dev-dependencies]`: `pretty_assertions`, `pi-test-utils`, `tokio-util`. Eight lost their last user in #26 (the session store). `pi-fast-worktree` lost its last users in `session::worktree` and `util::config::worktree` (this branch), and `tokio-util` in the tests of `session::worktree`. The dependency graph of `pi-pager-bin` goes from 980 to 970 packages on Linux: `pi-workflow` (a crate of this workspace; no other crate depends on it now) and nine packages from the registry (`tdigests`, `rhai`, `rhai_codegen`, `smartstring`, `thin-vec`, `const-random`, `const-random-macro`, `crunchy`, `tiny-keccak`). `Cargo.lock` loses `tdigests` and ten dependency lines. The other packages stay in the lock file, because `pi-workflow` is still a member of the workspace. `THIRD-PARTY-NOTICES` loses 5 entries (it lists the packages that ship on any platform; `tui_notices.py check`: 984 of 984). * chore(tui): lower the ceilings of the baseline after the cleanup of pi-shell - `[gates.residue]`: `leader` 245 -> 243 and `x.ai` 304 -> 297. The deletions of #26 removed those lines; `check` has said "lower the ceiling" since then. - `[graph.max_nodes]`: 980 -> 970 packages on Linux (the dependency commit of this branch). - `pi-shell` `reference_passed`: 864 -> 838. The tests of `session::worktree` (24, Unix only), `session::helpers` (2) and `util::config::worktree` (1) are gone, and one test is new (a `[storage]` section is reported as unrecognized). The ten references add up to 8,488 (8,514 before). A local run on Linux (WSL2) gives: `check` with 0 warnings, 970 packages, `leader` 243 and `x.ai` 297; the ten test suites 8,487 passed and 1 failed (the known `tmux_probe` test of `pi-pager-render`); `tui_notices.py check` 984 of 984. * docs: record plan r16 (the merge of #25 and #26, and what the session store left in pi-shell) - CHANGELOG.md: the entry for the third cleanup of pi-shell (15 files, three modules, the [storage] setting, 10 dependencies). It also says that the earlier entry left `[storage] cleanup_ttl_days` for this one. - docs/baselines/tui.md: current baseline r16 (tests 8,488, graph 970, `leader` 243, `x.ai` 297) and a column r16 in the table of sizes. The numbers are from local runs (WSL2); the CI numbers of PR #27 follow in a later commit. - docs/baselines/archive/tui-history.md: the r16 paragraph, with the CI numbers of `7d82c27`. - docs/baselines/tui-support.md: made again (`pi-shell` has 7 dead units, 1,008 lines, not 9 and 1,388). - Plan: status r16, next steps, decision 10, D.2, D.8 (new candidates: 7 dead units, the crate `pi-workflow`, `Config.storage_mode`, about 88 files that no code loads), D.9, a new open item "races between tests", the index and the revision list. - Execution log: section 10.19, B.11 (how to repeat the checks) and the r16 revision entry. * docs: record the CI result of PR 27 in the plan, the log and the baselines `CI` (run 39193752450) and `TUI CI` (run 39193752451) pass on 6bd7db5. `TUI CI`: 8,488 passed, 0 failed, 21 ignored; 970 packages in the graph; `leader` 243, `x.ai` 297; 0 warnings. These are the numbers that section 10.19 of the execution log expected, so only the "to be added" line and the timing bullet change, plus one sentence in each r16 entry. The cargo test job took 17 m 34 s and the gates job 4 m 24 s. The tests themselves took about 54 s of that time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plan decision 10, storage-layer part (answered by the owner on 2026-10-18: do it; the answer named only the storage layer).
pi-shellstill held the session store of the old Rust agent (session::storage,session::persistence). Since ADR1's PR 3 the pager reads no session file, so no code inzypicalls it. This PR deletes it, and the code that only it used. No behaviour changes in production code.The branch also has one test-only commit (
332ba75, 5 lines): a race between two tests ofpi-shellthat the full test run of this branch showed on every run. The race is older than this PR. See "A race between two tests" below.Stacked on #25 (base
fix/write-session-at-first-message). Not merged.Decision 10 has two more parts that this PR does not touch (the answer did not name them):
load_mcp_servers/util/config/mcp.rs, andpi-pager/src/recent_dirs.rs.A race between two tests
session::worktree::tests::resolve_worktree_by_id_or_path_existing_dir_returns_pathfailed in the full--librun ofpi-shellon this branch:failed to init worktree DB schema,disk I/O error,Error code 1802: Error invoking fstat to get file metadata.util::config::consent::tests::set_consent_answer_is_monotonic_per_accountsetsGROK_HOMEto its own temporary directory and deletes the directory at its end. Itsserialkey was the named keyGROK_HOME. Every other test that setsGROK_HOMEuses the default#[serial]key, and so do the twosession::worktreetests, which openworktrees.dbin whateverGROK_HOMEholds.serial_testruns tests with different keys at the same time, andEnvGuardhas no lock of its own. When the two overlap, the worktree test createsworktrees.dbin the temporary directory of the consent test, the consent test deletes it, and SQLite fails when it opens the WAL file: it callsstaton the database path to copy the permissions, gets ENOENT, and returns error 1802. The name of the code (SQLITE_IOERR_FSTAT) is misleading: the failing call isstaton a path, notfstat.LD_PRELOADtrace of a failing run (open, close, unlink, rename, stat, fstat, each with a backtrace): the worktree test's thread opens<tmp>/worktrees.db; 139 ms later the thread of the consent test deletes it; at 297 ms the worktree thread callsstat64on it twice and gets ENOENT, with the backtracegetFileMode<-findCreateFileMode<-unixOpen<-sqlite3WalOpen. Nofstatcall fails.session::worktree remote:: util:: config::tests, 12 threads) on the test binary ofc74cc18(the head of fix(agent): write a session file with its first message (plan decision 17) #25) fails 15 of 15 times. This PR only changes which tests run together in a full run.#[serial_test::serial], like the other tests that setGROK_HOME. One attribute and a comment; no production code.5a39e09)332ba75)pi-shell--lib, 20 runsThe known flaky test
auth::manager::lock::tests::dropping_the_guard_silences_the_heartbeat_before_anyone_else_can_hold_the_lockalso fails now and then in whole runs (5 of 15 onc74cc18; 2 and 1 of the 20 runs above). It is not related.Evidence
c74cc18(head of fix(agent): write a session file with its first message (plan decision 17) #25):gates.session_disk_reads0 hits, 980 packages in the graph, 0 warnings, 8,706 tests in the manifest.After:
scripts/tui_baseline.py check(WSL2) on5a39e09(3 m 19 s) and on332ba75(2 m 46 s), both pass: consumer build 0 warnings,--workspace --tests0 errors and 0 warnings, graph 980,leader243 (ceiling 245),x.ai297 (ceiling 304), no session-file reads in the pager.scripts/tui_notices.py check: 989 of 989 (Cargo.lockdid not change).scripts/tui_baseline.py test, WSL2, 16 m 24 s) on332ba75: 8,513 passed, 1 known failure (the tmux probe ofpi-pager-render, counted as passed by the manifest), 21 ignored.pi-shell864,pi-pager5,649, the other suites equal theirreference_passed. The run on the earlier work-in-progress commit43f291f(19 m 28 s) gave the same counts.zypiunder a pseudo-terminal (mock model):resume_matrix.py --strict16 PASS / 0 GAP / 0 FAIL,smoke.py23/23,exit_matrix.py12/12,print_exit.py,mode_probe.py,status_line_probe.py4/4,net_audit.py --mock13/13. The binary was built from2f33815.2f33815,5a39e09and332ba75have the same production code: they differ only inside#[cfg(test)].session::title(14 one-line changes, each run againstcargo test -p pi-shell --lib session::title): 13 are caught by 1 to 4 tests each. 1 survives: the cap of 100 scalars changed to 101 (the test uses the constant itself). The value 100 is the value of the old code.pytest(real_llmdeselected), same counts as before this PR (it adds no Python test): Windows 2,559 passed, 39 skipped, 31 deselected; Linux (WSL2,rgonPATH) 2,568 passed, 30 skipped, 31 deselected.ruff check .andruff format --check .pass.69932cf; it differs from332ba75by documents only; a later commit adds documents again):CI: Ruff passes;pyteston Python 3.11, 3.12 and 3.13, each 2,559 passed, 39 skipped, 31 deselected.TUI CI:Gates + cargo check3 m 41 s (graph 980 of 980, 0 warnings,leader243 of 245,x.ai297 of 304, no session-file reads in the pager);cargo test16 m 14 s, 8,514 passed, 0 failed, 21 ignored (pi-shell864,pi-pager5,649,pi-pager-render1,092: the tmux probe passes on the runner, so it has 1 more than the local run).TUI CIis no proof of the fix. The 12-thread numbers above are.zypi(Rust and PTY runs are Linux only); the one deleted test that compiled only on macOS was not run there. Other places with the same pattern as the race above:serial(GROK_HOME)inpi-pager(3 tests) and the other named keys inpi-shell(force_dark_wake_env,attribution_emit_count,heap_profile_hooks) were not checked for reads ofGROK_HOME.Merge Danger
Door: two-way
The change only deletes code that nothing calls, and moves three functions and one enum with their bodies unchanged. A revert brings the code back. No data, no wire format and no setting changes. The test-only commit changes the order in which tests may run; it does not touch production code.
Blast Radius: the
tui/tree ofpi-shell; people who build onpi_shell::session::{storage, persistence}pi_shell::session::storageorsession::persistencestops compiling. Nothing in this repository does: the consumer build andcargo check --workspace --testspass with 0 warnings.gates.session_disk_readsstays. Its 8 patterns now guard against writing a session store back into the pager.[storage] cleanup_ttl_daysis still accepted in the configuration, but no code reads it. It cleaned the old session files. A later change removes it.pi-shellare no longer named in its sources (urlencoding,pi-compaction-transcript,pi-workflow,tdigests,nix; dev:pretty_assertions,pi-test-utils). A text search found them; macros and renames can fool it. They are not removed here.session::helpers(21 lines) andsession::worktree(359 lines) as dead. rustc gives no warning for them. They are left for plan D.8.serialtests ofpi-shell(about 90 of them). The lib tests ofpi-shelltook 8.2 s in the whole run on332ba75, as before (8 to 9 s).mainand mergemaininto its branch (git merge -s ourswhen the trees are equal), as for refactor(tui): delete the disk-reading session code (ADR1 PR 3) #24.