Skip to content

Commit 4fc028a

Browse files
authored
feat(tui): plan 0.5 call graph tool and phase C closeout (acceptance probes, 7-page guide, notices; plan r11) (#13)
* feat(scripts): add tui_callgraph.py, the module-level call graph of the TUI workspace (plan 0.5) The tool reads the Rust sources and reports which modules live code refers to. It replaces the hand-written list of examples of what the support layer keeps. Subcommands: crates, units, keep. 35 tests; each mutation of the tool fails them. * feat(tui): keep THIRD-PARTY-NOTICES in step with Cargo.lock, rewrite NOTICE (plan C.3) scripts/tui_notices.py finds the packages that ship in zypi (normal and build dependency closure of pi-pager-bin, five targets) and fixes the notices file: 222 stale entries out, 44 missing entries in, hand-written sections kept. The wording of a new entry is copied from the entries with the same license. tui/NOTICE now names the product (zypi), says tui/ is a hard fork that does not follow upstream, and lists the changes. The stale nfsserve entry in third_party/ is gone. 62 tests. * refactor(tui): C.2 - embedded user guide cut to the zypi pages, tutorial overlay removed, no per-start extraction - docs: seven pages (getting started, slash commands, theming, sessions, terminal support, permissions and safety, status line) written against the code and agent.toml; sixteen pages of removed features deleted. The status-line page says the row stays empty with the Python agent (it is filled from a session-status update only the Rust agent sent). - docs.rs: USER_GUIDE lists the seven pages; extract_user_guide_docs and its call in pi-pager-bin are gone; two tests guard the pages (no old product names, every file in the directory is listed). - tutorial overlay (views/tutorial.rs, tutorial_docs.rs, docs/tutorial/) removed: no entry point since A.3, and its pages described commands zypi does not have. - doc picker: the footer tip pointed at files zypi no longer writes. - strings that pointed at ~/.grok/docs/user-guide/*.md (mcp.rs, glob.rs), the zypi wrap help line that pointed at a README that does not exist, and two comments. * test(tui_pty): net_audit, status_line_probe and status_proxy_agent (plan C.1) net_audit.py plays one turn under a pseudo-terminal and samples the sockets of zypi and of each process below it (/proc/<pid>/fd joined with /proc/<pid>/net/*). zypi must hold no connection to a non-loopback address and listen on none. --mock checks that the agent is silent too, --no-network runs it in an empty network namespace, and --model-from FILE plays one real turn (the agent must hold an established connection: the positive control; the output names no host, address or key). status_line_probe.py shows that the status row is not drawn with the plain Python agent and is drawn when status_proxy_agent.py adds the session_status message. Unit tests for the three run on Windows and Linux. * docs: C.4 - config.toml constraint measured, TUI guide corrected, Phase 4 spec revised, P2 admission rule in DESIGN AGENTS.md (pi-agent-cli) says what zypi really does with config.toml: strict types for the keys it knows (a top-level permission string, a [[models]] array and invalid TOML stop it with exit 1), unknown keys ignored; the agent falls back to config.toml only when agent.toml is missing. TUI-AND-CODE-AGENT.md: ACP link, real [model] keys (id, not model_id; no context_window; permission values ask|auto|always-approve; [skills] paths), the real slash commands, no model client / tool runner / MCP client in zypi, pointer to the network audit. Phase 4 spec section 1.3 #2 and section 3 get a revision note (runtime part of the shell is gone; pi-tools and pi-workspace still tie the pager to the tree; tui/ is a hard fork). DESIGN section 8.5 states the five-condition admission rule for pi/ extensions (plan P2). * chore(tui): clear the stale 'grok agent' pointers and ratchet the leftover words (plan C.4) The pi-shell README was the upstream Grok CLI manual (1,968 lines, an install script that installs the official CLI): replaced by a short note on what the crate is now. Doc comments and a label in pi-agent, pi-http, pi-tty-utils and pi-test-support no longer name a 'grok agent' command. pi-test-support's README now says that only EnvGuard, scaled, TestProcess/TestSandbox, env_parse and ResourceGrowth have users outside the crate (plan D.5). tui_baseline.py gets count_residue and a [gates.residue] ceiling per phrase: 'grok agent' 0, 'leader' 245, 'x.ai' 304 lines under tui/crates (the plan's literal zero cannot be met before phase D: the lines are the removed leader process, xAI endpoints in the auth, voice and update code, and the vendor-method filter itself). A test runs the same ratchet in the Python CI job. * docs(tui): add the generated keep list and the leftover-word table, set the pi-pager baseline to 5,608 tui-support.md is the output of tui_callgraph.py keep (item 0.5). The tui.md section on leftover words classifies the 245 leader and 304 x.ai lines that the new residue gate holds as ceilings. pi-pager is 5,626 -> 5,608 after the embedded user guide change (item C.2); the other nine suites did not move (8,665 over ten suites, one local run on Linux). * fix(scripts): make tui_callgraph.py precise enough to trust its dead list (plan 0.5) Check: copy tui/, delete the files of every unit the tool calls dead, run cargo check on the consumer crates. The first run listed 21 dead units and 4 of them were live: - a path inside a string of an attribute (`serde(with = "crate::util::serde_base64")`): `util::serde_base64`, `serde_helpers` - a path that starts at a child module, with no `use` and no `crate::` (`hub_auth::init_metrics()` in lib.rs): `hub_auth`, `hub_auth::proactive` Both are references now. A third kind cannot be seen from the text: an impl that code uses without a name (`types::error` holds the `impl From<TodoError> for ToolError` that `?` calls). A dead unit with `impl` blocks for types it does not define is listed apart as "check first", and the summary table gets a column for it. Result on 24907ef: 17 dead units, 6,190 lines. With all 17 deleted, cargo check fails at exactly one unit (`types::error`, 5 errors, and it is in the "check first" list); with that one kept, the check passes and shows one new warning (an unused function in pi-hooks, which is a function, not a module). Tests 35 -> 39. Each of the 17 mutations of the tool makes a test fail. docs/baselines/tui-support.md is regenerated. * docs: record r11 (call graph tool, phase C) in the plan, execution log, baselines and changelog Plan: 0.5 and C.1-C.4 are done; decision 15 (the status line) and the phase D candidates D.5-D.9 are new; index, status and revision text say r11. D.9: the Rust test run does not isolate HOME, so some tests write the developer's real ~/.pi-python and /tmp. Execution log: section 10.14, appendix B.7 and the revision log. Baselines: tui.md gets the r11 numbers and a keep-list section; tui-history.md gets the r11 paragraph. CHANGELOG: the call graph tool, the notices check, the network audit, the status line probe, the 7-page user guide, NOTICE, the leftover-word ratchet, the config.toml notes. AGENTS.md: the two new TUI tools, the new PTY scripts and the HOME warning. Numbers: 10 suites 8,665 passed / 0 failed / 22 ignored (WSL2, one local run); pytest Windows 2,510 and Linux 2,538 passed; Ruff 0.16.10 and 0.16.0 clean. Nothing is pushed.
1 parent df812d1 commit 4fc028a

87 files changed

Lines changed: 6366 additions & 15659 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ AgentMessage[] → transform_context() → convert_to_llm() → LangChain BaseMe
5050

5151
## Development
5252

53-
Python monorepo (`pi-agent-core`, `pi-agent-harness`, `pi-agent-cli`) + Rust TUI workspace `tui/` (vendored grok-build fork, binary `zypi`). Packages installed editable; tested via `pytest`. TUI: `cd tui && cargo check -p pi-pager-bin` (WSL; Cargo reads `CARGO_TARGET_DIR` from WSL env exclusively). Rust tests run through `python scripts/tui_baseline.py test` (manifest `scripts/tui_baseline.toml`, numbers in `docs/baselines/tui.md`); it clears `PI_HOME` and friends, which a WSL login shell may export pointing at the real home — do not run test binaries or `zypi` without clearing them, and build on the Linux file system, not `/mnt/<drive>`. With a built `zypi`, `scripts/tui_pty/` drives it under a pseudo-terminal on Linux (smoke test, exit matrix, `-p` shutdown, the matrix of flags that resume a session, a probe of which permission modes the agent honours; README there). The pager's ACP client is also run against the real Python agent without a terminal: `tui/crates/codegen/pi-pager/tests/python_agent_e2e.rs`, which needs `PI_E2E_PYTHON` (a Python with `pi_agent_cli` installed; TUI CI sets it and `PI_E2E_REQUIRED=1`, locally an unset variable makes the tests skip with a note).
53+
Python monorepo (`pi-agent-core`, `pi-agent-harness`, `pi-agent-cli`) + Rust TUI workspace `tui/` (vendored grok-build fork, binary `zypi`). Packages installed editable; tested via `pytest`. TUI: `cd tui && cargo check -p pi-pager-bin` (WSL; Cargo reads `CARGO_TARGET_DIR` from WSL env exclusively). Rust tests run through `python scripts/tui_baseline.py test` (manifest `scripts/tui_baseline.toml`, numbers in `docs/baselines/tui.md`); it clears `PI_HOME` and friends, which a WSL login shell may export pointing at the real home — do not run test binaries or `zypi` without clearing them. The run does not isolate `HOME` (plan D.9): some tests write into the real `~/.pi-python` and `/tmp`, so check both after a local run. Build on the Linux file system, not `/mnt/<drive>`. With a built `zypi`, `scripts/tui_pty/` drives it under a pseudo-terminal on Linux (smoke test, exit matrix, `-p` shutdown, the matrix of flags that resume a session, a probe of which permission modes the agent honours, a network audit (it shows that `zypi` itself opens no connection to a non-local address during a turn), a probe of the status line; README there). Two tools read the TUI tree: `python scripts/tui_callgraph.py keep --out docs/baselines/tui-support.md` regenerates the list of what the support crates still use (it reads source text only; the list is an estimate, `cargo check` has the last word), and `python scripts/tui_notices.py check` (needs cargo) says whether `tui/THIRD-PARTY-NOTICES` still matches `Cargo.lock`; run it after a dependency change, then `update --write`. The pager's ACP client is also run against the real Python agent without a terminal: `tui/crates/codegen/pi-pager/tests/python_agent_e2e.rs`, which needs `PI_E2E_PYTHON` (a Python with `pi_agent_cli` installed; TUI CI sets it and `PI_E2E_REQUIRED=1`, locally an unset variable makes the tests skip with a note).
5454

5555
### Virtual environments (uv)
5656

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,16 +15,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1515
- **`session/list` comes in pages (`pi-agent-cli`, `pi-agent-harness`)**: the agent answers with at most 50 sessions, newest first, and a `nextCursor` while more follow; the cursor is opaque, carries the position in it (the agent keeps nothing between requests, so it survives a restart) and anything the agent did not issue is `invalid_params`. The TUI keeps asking until the cursor is gone, so its session picker still lists everything. For library users `JsonlSessionRepo.list_page(...)` returns one page and `JsonlSessionRepo.find(id)` reads one session's header instead of the directory. With 1,000 sessions of 120 entries on Windows, the first answer took 2.0 s and a `session/load` of one of them 2.0 s; reading a file's first line no longer reads the file and a lookup by id no longer lists the directory, which makes them 0.11 s and 0.12 s (`scripts/session_list_bench.py`; numbers and method in `docs/PLAN/archive/PLAN-RUST-AGENT-RUNTIME-REMOVAL-EXECUTION-LOG.md` §10.11). `updated_at` is still the file's mtime and need not follow the list's order, which is by creation time.
1616
- **The agent says which MCP servers a session ignored (`pi-agent-cli`)**: the agent connects to no MCP server (a Phase 4 non-goal), which departs from ACP's "an agent MUST support stdio servers"; `session/new`, `load` and `resume` used to drop the list without a word. Each session set up with a non-empty list now logs a warning and sends one agent message naming the servers (names only: `env`, `headers`, `args` and `url` hold tokens). `zypi` no longer forwards what it finds in `.mcp.json`, `~/.claude.json` and Cursor's `mcp.json`, so there the message never shows; it is for other ACP clients, an editor that passes its own servers, say.
1717
- **A client can choose the id of a new session, and the agent says which directory a session works in (`pi-agent-cli`, `pi-agent-harness`)**: `zypi --session-id <uuid>` has always sent the id in `session/new`'s `_meta.sessionId`; the agent ignored it and picked its own, so the flag did nothing on the Python agent. It now becomes the session's id. It must be a UUID in any form `uuid.UUID` reads (kept as lower case with hyphens); anything else is `invalid_params`, because the id ends up in a file name. An id some session already has is refused with `invalid_params`, whatever directory that session belongs to, and two requests for the same id at once make one session. `session/new`, `load` and `resume` answer with `_meta["pi/cwd"]`, the directory the session works in; for `load` and `resume` the one saved with the session wins over the `cwd` the client sent. For library users `JsonlSessionRepo.has_session(id)` says whether an id is taken and opens only the files whose names do not already say they belong to another session (`find` reads every header for an id nobody has). A free id costs the directory listing: 0.07 s at 1,000 sessions, 0.3 s at 5,000, on Windows and WSL2 alike (`scripts/session_list_bench.py`). **Not yet in the TUI**: `zypi` still checks for a used id in the old session layout, so it never sees the Python agent's sessions; the agent refuses the id and the TUI shows `Session creation failed: Invalid params: {...}` instead of exiting 1 with a message (`scripts/tui_pty/resume_matrix.py`, case 11), and nothing reads `pi/cwd` yet (execution log section 10.11, ADR1 PR 2).
18+
- **A call graph of the TUI workspace (`scripts/tui_callgraph.py`)**: it reads the Rust source as text, so it needs no `cargo` and no network, and it gives the same answer on Windows, Linux and macOS. `crates` shows the dependency closure of the product crates (78 crates in the workspace, 74 in the closure; `--without pi-shell` shows what only `pi-shell` keeps alive). `units` shows, for one or more crates, the modules that no live code refers to; `--ban` asks what dies if a module goes. `keep` writes `docs/baselines/tui-support.md`, which replaces the hand-written list of examples of what the support crates keep. It says: 305,581 lines of non-test code in the 7 support crates, and 17 modules (6,190 lines, 4,339 of them in `pi-hooks`) that no live code refers to. A module on that list is a candidate for deletion, not proof, so the list was checked with the compiler. In a copy of `tui/`, 16 of the 17 modules were deleted, and `cargo check` passed (also `cargo check --workspace --tests`, after the test file of the removed `pi-hooks` modules went too). The 17th, `types::error` in `pi-tools`, cannot go: it holds the `impl From<…> for ToolError` that `?` calls, and no path names it. The tool marks such modules "check first". The first version listed 21 modules and 4 of them were live: it did not see a path inside an attribute string (`serde(with = "crate::util::serde_base64")`) or a path that starts at a child module. Both are fixed. 39 tests; each of 17 changes made to the tool makes a test fail. Nothing was deleted from `tui/` (plan item D.8; execution log section 10.14).
19+
- **A check that `tui/THIRD-PARTY-NOTICES` matches the dependencies (`scripts/tui_notices.py`)**: the file came from the upstream project and did not follow `Cargo.lock`. `check` finds the packages that ship in `zypi` (the normal and build dependencies of `pi-pager-bin`, for five target triples, from crates.io, git or `tui/third_party/`) and exits 1 if an entry is missing or stale. `update --write` fixes the file: it removes stale entries, adds missing ones and leaves the hand-written parts and the license texts alone. The license wording of a new entry is copied from the entries with the same SPDX expression; a package with no such entry is listed as `needs review`. First run: 222 stale entries removed (packages that are not in the binary) and 44 missing entries added, so there are 989 entries for 989 packages, and the file went from 18,898 to 16,548 lines. None of the 44 needed review. This matches the file to `Cargo.lock`. It is not a legal review: nobody has read the copied wording one entry at a time. `check` needs `cargo`. 62 tests.
20+
- **A network audit of `zypi` (`scripts/tui_pty/net_audit.py`, Linux)**: it runs `zypi` in a pseudo-terminal, plays one turn, and reads the sockets of `zypi` and of every process below it about every 9 ms (`/proc/<pid>/fd` joined with `/proc/<pid>/net/{tcp,tcp6,udp,udp6}`). `zypi` must hold no connection to a non-loopback address and must listen on none. With the real model, the agent must hold such a connection (the positive control: it shows that the sampler can see a connection). Result: 13 of 13 checks with the mock LLM; 13 of 13 with the whole process tree in an empty network namespace (`--no-network`, `bwrap --unshare-net`), and the turn still ends normally; 13 of 13 for one turn with the real model of the machine. The output names no host, address or key, and the key goes to the agent through an environment variable only. A connection that opens and closes between two samples is not seen.
21+
- **A probe for the status line (`scripts/tui_pty/status_line_probe.py`, `status_proxy_agent.py`, Linux)**: it shows that `[ui.status_line]` (`type = "command"` or `"builtin"`) draws nothing with the Python agent, and that the command script is not run. The row is built from a `session_status` notification that only the old Rust agent sent. `status_proxy_agent.py` runs the real agent and adds that one notification after each turn; with it, both types draw the row (4 of 4 runs). What to do about it is plan decision 15.
1822

1923
### Changed
2024

2125
- **`zypi` offers only the permission modes the agent honours (TUI)**: the agent asks about tool calls or approves them all, and nothing in between, but the TUI still offered Plan, Auto, a settings "Default" and six `--permission-mode` values, none of which did what it said. Plan restricted nothing (the agent has no `session/set_mode` handler, so the call failed unseen while the screen said `plan`); Auto never asked, because the agent treats `auto` like `always-approve` while the screen promised a classifier; the settings' Default is a value the agent drops, so after Always-Approve it said Default and the agent still never asked; and of the six `--permission-mode` values only `bypassPermissions` did anything, the other five behaved exactly like no flag (`acceptEdits` accepted no edits, `dontAsk` asked all the same). Now Shift+Tab toggles Normal and Always-Approve; the `plan_mode` setting, the plan-mode tip, and the Default and Auto choices of `permission_mode` are gone from the settings page; and `--permission-mode` takes `default` (no flag) and `bypassPermissions` (`--always-approve`) only. **Behaviour change**: `--permission-mode plan`, `auto`, `acceptEdits` and `dontAsk` now fail at start-up with exit status 2 and a message naming the value and the values that work (they used to start a normal, asking session); a saved `permission_mode` of `auto` or `default` is shown as Ask and the file is left alone; `--permission-mode` used in a script needs `default` or `bypassPermissions`. The code behind the hidden modes stays, behind three constants in `app/agent_modes.rs`; flip one when the agent honours that mode. Measurements before and after: `docs/PLAN/archive/PLAN-RUST-AGENT-RUNTIME-REMOVAL-EXECUTION-LOG.md` sections 10.11 and 10.13 (`scripts/tui_pty/mode_probe.py`).
26+
- **The user guide inside `zypi` has 7 pages, and `zypi` no longer writes it to disk at start-up (TUI)**: the 33 files under `pi-pager/docs/` (23 user-guide pages, a README and 9 tutorial pages) were the upstream text for a different program. They named the `grok` command, `grok agent`, MCP servers, plugins, the leader process and other features that `zypi` no longer has. Now `user-guide/` holds a README and 7 pages written against the code and `agent.example.toml`: getting started, slash commands, theming, sessions, terminal support, permissions and safety, status line. The tutorial overlay (`views/tutorial.rs`, `tutorial_docs.rs`) is gone: nothing opened it since the entry-point audit. `zypi` no longer copies the pages to `<home>/docs/user-guide/` at each start (`extract_user_guide_docs`); copies that earlier versions wrote stay where they are. `pi-pager` tests go from 5,626 to 5,608: the tests of the removed pages and overlay go, and two guard tests are new (no old product name on a page; every file of the directory is in the list built into the binary). Nothing opens the document picker and viewer now, so the 7 pages have no reader until plan item D.6 is decided.
27+
- **`tui/NOTICE` names the product**: it said `pi` (`pi-grok-pager-bin`). Now it says `zypi`, says that `tui/` is a hard fork that does not follow upstream, and lists the changes. The stale `nfsserve` entry in `tui/third_party/` is gone.
28+
- **Leftover words in the TUI sources are held by a ratchet (`scripts/tui_baseline.py`, `[gates.residue]` in `scripts/tui_baseline.toml`)**: each word has a ceiling on the number of lines under `tui/crates`, and the ceiling can only go down. Now: `grok agent` 0, `leader` 245, `x.ai` 304. The plan asked for zero. That cannot be done for `leader` and `x.ai` before phase D: the lines are the name of the removed leader process, xAI service addresses in the auth, voice and update code, and the filter that drops `x.ai/*` extension notifications. `docs/baselines/tui.md` section 4 says what each group is and where it goes. A Python test runs the same check, so the Python CI job catches a rise too. All 28 lines with `grok agent` are gone: 15 were in user-guide pages and 13 in doc comments, a label and two READMEs (`pi-shell/README.md` was the upstream user manual, 2,682 lines; it is now a 43-line note on what the crate is).
29+
- **What `zypi` does with `config.toml` is written down (`packages/pi-agent-cli/AGENTS.md`, `docs/WINDOWS.md`, `docs/TUI-AND-CODE-AGENT.md`)**: the old text said that Python-only keys break TUI parsing. Measured: `zypi` is strict about the keys it knows and ignores the keys it does not know. A missing file, an unknown table or key and a Python `[model]` table start normally. A top-level `permission = "ask"` (`zypi` expects a table there), a `[[models]]` array and invalid TOML stop `zypi` with exit status 1. The agent reads `agent.toml`, and falls back to `config.toml` only when `agent.toml` does not exist. `TUI-AND-CODE-AGENT.md` now has the real `[model]` keys (`id`, not `model_id`; no `context_window`), the permission values (`ask`, `auto`, `always-approve`) and the real slash commands, and says that `zypi` has no model client, tool runner or MCP client. `docs/DESIGN.md` section 8.5 states the five conditions for a new `pi/` extension, and the Phase 4 spec has a revision note.
2230

2331
### Fixed
2432

2533
- **`zypi` told the user nothing when the agent failed to start or died**: a missing agent program surfaced as `channel closed ... recv_failed`, and an agent that exited as soon as it started (a Python without `pi_agent_cli`, say) left `connect` waiting forever; an agent that died in the middle of a turn left the requests in flight waiting too. The stdio bridge now notices the exit and says what happened: the program, its exit status and the last lines the agent wrote to `agent.stderr.log` (`failed to spawn ACP agent ... PI_AGENT_COMMAND` for a program that is not there). `zypi`'s exit status is unchanged. Found by the new end-to-end suite that runs the pager's ACP client against the real agent (`tui/crates/codegen/pi-pager/tests/python_agent_e2e.rs`; TUI CI installs the agent for it).
2634
- **`zypi -p` left the agent and its tools running when only `zypi` was killed**: `kill <pid>`, `kill -9`, `timeout` or an IDE's stop button reached `zypi` but not the agent, which kept running (and kept running its `bash` call) on the user's terminal. `zypi` now passes its pid in `PI_AGENT_PARENT_PID`; `python -m pi_agent_cli -p` watches its parent and stops the way it stops on a signal (turn cancelled, tools reaped), and `-p` now handles SIGTERM and SIGHUP too (exit status 128 + the signal number). POSIX only; the variable is removed from the environment at start-up, so tools do not inherit it, and it is ignored when it names a live process that is not the agent's parent (a wrapper script).
2735
- **`zypi --sandbox ...` on Linux left the whole process tree running when only the outer `bwrap` was killed**: the sandbox re-launches `zypi` under `bwrap`, which by default just waits for its child, so signalling the process the shell started (`kill $!`, a supervisor) left `zypi`, the agent and its tools on the terminal. `bwrap` now runs with `--die-with-parent`. **Trade-off**: a SIGTERM or SIGHUP sent to that outer `bwrap` now reaches `zypi` as SIGKILL, so the terminal is not restored (alternate screen, raw mode); before, the tree kept running instead.
36+
- **Messages in `zypi` that pointed at files that do not exist**: five messages (four from the MCP config code, one from the sandbox glob code) ended with `See ~/.grok/docs/user-guide/<page>.md`, a path that `zypi` never uses; the help of `zypi wrap` ended with `See ~/.pi-python/README.md for more information.`, and there is no such README; and the footer of the document picker named the folder where `zypi` used to write the pages. The pointers are gone.
2837

2938
## [0.5.0] - 2026-10-02
3039

0 commit comments

Comments
 (0)