Skip to content

Commit ec7d455

Browse files
authored
feat(tui): zypi export reads the session through the agent (plan decision 6) (#18)
## Summary `zypi export <id>` read the session files of the old Rust agent, so it failed for every session that the Python agent stored. It now asks the agent, as a resume does. This is plan decision 6, option (a). ```text zypi export <id> connect() # starts the Python agent session/load(id, cwd) collect session/update (isReplay) done when: the answer came AND no replayed update came for 500 ms before the answer: 60 s of silence is an error AcpUpdateTracker -> ScrollbackState -> render_blocks_to_markdown # same path as /export AgentProcessGuard drop: cancel + join # also when the command fails ``` ```diff pi-pager/src/ -├── export_cmd.rs # read the old Rust session store +└── export_cmd/ + ├── mod.rs # entry and rendering + ├── replay.rs # agent and collection of the replayed updates + └── tests.rs # 18 tests, paused tokio clock ``` The answer to `session/load` can come before the last replayed messages. So the answer alone does not end the collection: only the quiet period does, and only a replayed update restarts it. Permission requests get `Cancelled`, other requests from the agent get method-not-found. `isReplay` is read from the notification `_meta` and from the update `_meta` (the Python agent sets it on the update). ## Evidence - **Before:** `resume_matrix.py` on `main` (plan section 10.12): 10 PASS / 3 GAP. Case 12, `zypi export <id>`, was GAP: `Session '<id>' not found.`, exit 1. **After:** 11 PASS / 2 GAP / 0 FAIL on WSL2. Case 12 passes. Case 13 (unknown id) passes as before. The two GAP that remain are `--resume <title>` and `--session-id <id in use>`; this PR does not touch them. Cases 12 and 13 now also fail when a process is left behind. - **Time** (debug build, WSL2; three runs each, the session file is not changed): | Entries | File | `zypi export` | Agent alone (`session/load`) | |---|---|---|---| | 2,000 | 1.4 MB | 2.5 s | 0.24 s | | 5,000 | 3.6 MB | 5.6-5.7 s | 0.53 s | | 20,000 | 14.2 MB | 62.7-64.1 s | 2.5 s | The agent is fast. The rest is `zypi` turning the messages into text, and that grows faster than the size. A release build was not measured. - **Tests:** 18 unit tests in `export_cmd::tests`. 15 run on a paused tokio clock: they cover the order of the updates, the quiet period, the silence limit, requests from the agent, refusals, and an agent that exits or says nothing. 3 cover the rendering. 1 end-to-end test, `export_replays_a_stored_session_through_an_agent_of_its_own`, against the real Python agent; `python_agent_e2e` passes 11 of 11. 1 new test in `scripts/tests/test_tui_pty.py`. - **Checks that ran locally** (on the code of the final commit; the last edits were documentation only): `tui_baseline.py check` exit 0 (warnings 0, residue words unchanged, dependency graph 980 = ceiling, `cargo check --workspace --tests` 0 errors). `ruff check` and `ruff format --check` pass. `pytest`: Windows 2,517 passed, 39 skipped, 31 deselected; WSL 2,540 passed, 16 skipped, 31 deselected. - **CI** (on `e3f3458`: [TUI CI run](https://github.com/zy1233/pi-python/actions/runs/38344799030), [CI run](https://github.com/zy1233/pi-python/actions/runs/38344799403)): every job passes. `Ruff`. `pytest` 3.11, 3.12 and 3.13. `Gates + cargo check`: warnings 0, `leader` 245 of 245, `x.ai` 304 of 304, graph 980 of 980. `cargo test`: 8,684 passed, 0 failed, 22 ignored (`pi-pager`: 5,627 passed, 13 ignored). `Release build baseline` is manual and was skipped. - **Not verified:** Windows and macOS (only WSL2 ran `zypi export`). Release-build time. A real model (the agent is a mock; export does not call the model). - **Found, not fixed** (details in plan section 10.15, "continuation 6"): the Python agent loops forever on a session file with repeated entry ids; `scripts/session_list_bench.py` writes such a file above 4,096 entries; the pager reads `isReplay` only from the notification `_meta`, the agent sends it on the update (the resume path was not checked). ## Merge Danger **Door:** two-way The change reads session files and writes none. A revert restores the old behaviour, which fails for every session of the Python agent. **Blast Radius:** export `zypi export` now needs a working agent command, as a resume does; it did not before. A very large session takes about a minute on a debug build. Nothing else in the pager changes. The new `test-util` feature of `tokio` is a dev-dependency only. Merged on the owner's order, into V0.5 (`v0.5.0`). The CHANGELOG entry moves from `[Unreleased]` to `[0.5.0]`, and the `zypi export` line leaves the known issues, in the commit that follows (it is part of #19, which merges after this PR). `tui/` changed, so the release size is measured again on the final tree (`release_baseline`, condition 2) and CI must pass on the tag commit (condition 1).
1 parent 62b0a0c commit ec7d455

15 files changed

Lines changed: 825 additions & 52 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Fixed
11+
12+
- **`zypi export <id>` works with the Python agent (TUI)**: it read the session files of the old Rust agent, so it answered `Session '<id>' not found.` for every session that the Python agent stored. It now asks the agent, as a resume does: it starts the agent, sends `session/load`, collects the `session/update` messages that the agent replays, and writes the Markdown transcript as before (to stdout, to a file, or to the clipboard; plan decision 6, option (a)). The agent is stopped when the command ends, also when the command fails. The command does not stop at the answer to `session/load`, because the last replayed messages can arrive after it: it waits until no replayed message has come for 500 ms. An agent that says nothing for 60 s is an error (`it said nothing for 60 s`), and a permission request from the agent is refused. A session that was compacted exports from the compaction on, as a resume shows it. The session file is not changed. Time, on a debug build in WSL2: 2,000 entries (1.4 MB) 2.5 s, 5,000 entries (3.6 MB) 5.7 s, 20,000 entries (14.2 MB) 63 s. The agent alone needs 0.24 s, 0.53 s and 2.5 s for the same sessions, so the rest is `zypi` turning the messages into text, and that time grows faster than the size. Tests: 18 unit tests, 1 end-to-end test against the real agent, and `scripts/tui_pty/resume_matrix.py` cases 12 and 13, which now also check that no process is left behind.
13+
1014
## [0.5.0] - 2026-10-12
1115

1216
`zypi` no longer has an agent of its own. It draws the terminal screen and talks ACP to the Python agent (`python -m pi_agent_cli`), and that agent is now the only agent. **Removed** says what went with the Rust agent runtime. **Known issues** says what does not work yet. This release also has the Phase 6 and Phase 7 changes to the Python packages: project trust, a sandbox for workflow scripts, DeepSeek thinking with tools, and tool annotations.

‎docs/PLAN/PLAN-RUST-AGENT-RUNTIME-REMOVAL.md‎

Lines changed: 11 additions & 11 deletions
Large diffs are not rendered by default.

‎docs/PLAN/archive/PLAN-RUST-AGENT-RUNTIME-REMOVAL-EXECUTION-LOG.md‎

Lines changed: 21 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/baselines/tui-support.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
This file is generated. Do not edit it by hand.
44

5-
- Command: `python scripts/tui_callgraph.py keep`. Commit: `24907ef`.
5+
- Command: `python scripts/tui_callgraph.py keep`. Commit: `34bf8f6`.
66
- Run the command again after a change that moves code between modules or crates.
77
- It replaces the hand-written list of examples in item 0.5 of
88
[`PLAN-RUST-AGENT-RUNTIME-REMOVAL.md`](../PLAN/PLAN-RUST-AGENT-RUNTIME-REMOVAL.md).
@@ -54,7 +54,7 @@ Check these dead units first. Each has `impl` blocks for types that other units
5454

5555
| unit | lines | references from live code outside |
5656
|---|---|---|
57-
| `session::storage` | 7,372 | 1 |
57+
| `session::storage` | 7,372 | 0 |
5858
| `util::config` | 6,770 | 94 |
5959
| `auth::manager` | 3,754 | 0 |
6060
| `agent::config` | 3,167 | 26 |

‎scripts/tests/test_tui_pty.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,29 @@ def test_the_session_id_cases_tell_a_used_id_from_a_fresh_one():
230230
assert not taken.holds(started_as_another, IDS)
231231

232232

233+
def test_the_export_cases_tell_a_transcript_from_a_refusal():
234+
printed = next(case for case in rmx.CASES if case.argv == ["export", "{existing}"])
235+
unknown = next(case for case in rmx.CASES if case.argv == ["export", "{unknown}"])
236+
conversation = f"## User\n\n{rmx.FIRST}\n\n## Assistant\n\n{rmx.REPLY}"
237+
transcript = rmx.Outcome(0, "", stdout=conversation)
238+
half = rmx.Outcome(0, "", stdout=f"## User\n\n{rmx.FIRST}")
239+
agent_left = rmx.Outcome(0, "", stdout=conversation, left_behind=["python -m pi_agent_cli"])
240+
not_found = rmx.Outcome(1, "Error: Session 'u' not found.")
241+
# The export goes through the agent now, so neither case may carry a gap.
242+
assert not printed.gap
243+
assert not unknown.gap
244+
assert printed.holds(transcript, IDS)
245+
assert not printed.holds(half, IDS)
246+
assert not printed.holds(agent_left, IDS)
247+
assert not printed.holds(not_found, IDS)
248+
assert unknown.holds(not_found, IDS)
249+
assert not unknown.holds(rmx.Outcome(1, "Error: boom"), IDS)
250+
assert not unknown.holds(rmx.Outcome(0, "", stdout=conversation), IDS)
251+
assert not unknown.holds(
252+
rmx.Outcome(1, "Error: Session 'u' not found.", left_behind=["python"]), IDS
253+
)
254+
255+
233256
def test_sessions_reads_the_id_from_each_header(tmp_path):
234257
folder = tmp_path / "sessions"
235258
folder.mkdir()

‎scripts/tui_pty/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ a built `zypi`, so they are not part of `pytest` or CI.
1010
| `smoke.py` | Welcome page, one turn, `/model`, `/exit`, then a restart in the same home with `/resume`; the agent's stderr log (`$PI_HOME/logs/agent.stderr.log`, owner-only, appended per run); with `--sandbox PROFILE`, what the status bar says about the profile |
1111
| `exit_matrix.py` | While the agent runs a long `bash` tool call, leave zypi six ways (`/exit` twice, `kill -9`, `SIGTERM`, `SIGHUP`, closing the terminal, Ctrl-C) from two launch styles (zypi as session leader, zypi as a job of an interactive shell); nothing may be left running |
1212
| `print_exit.py` | `zypi -p ...` while only zypi gets `SIGTERM` / `SIGKILL` / `SIGHUP` / `SIGINT`: the agent and its tool call must stop too |
13-
| `resume_matrix.py` | What `--continue`, `--resume <id>`, `--resume <title>`, `--session-id` and `zypi export` do with a session the Python agent wrote (13 cases). Three are **known gaps** (reported as `GAP`, not failures; `--strict` fails them): the pager still reads the old Rust agent's session files for them (a fourth, `--session-id <fresh uuid>`, closed when the agent began to honour `_meta.sessionId`). Landing ADR1 (execution log section 10.11) has to close them and drop each case's `gap` text; a gap that has closed shows as `FIXED` |
13+
| `resume_matrix.py` | What `--continue`, `--resume <id>`, `--resume <title>`, `--session-id` and `zypi export` do with a session the Python agent wrote (13 cases). Two are **known gaps** (reported as `GAP`, not failures; `--strict` fails them): the pager still reads the old Rust agent's session files for them (`--session-id <fresh uuid>` closed when the agent began to honour `_meta.sessionId`; the two `zypi export` cases closed when `zypi export` began to start an agent and replay the session with `session/load`: they check the transcript on stdout, an error that names an unknown id, and that no agent is left running). Landing ADR1 (execution log section 10.11) has to close the two that are left and drop each case's `gap` text; a gap that has closed shows as `FIXED` |
1414
| `mode_probe.py` | Which permission modes the TUI offers (Shift+Tab cycle, `--always-approve`, `--permission-mode default\|bypassPermissions`) the Python agent honours: for each, whether a `write` is asked about and whether it is carried out unasked; then the four `--permission-mode` values that are refused (`acceptEdits`, `auto`, `dontAsk`, `plan`), each of which must exit at once with the reason; last, the settings page (does a search for "plan" find a "Plan mode" row, which choices does the Permission mode picker list). Judges nothing, prints a table (execution log section 10.13; plan appendix A, 1.P3 (a)): Plan, Auto and the settings page's Default are hidden because the agent honours neither (section 10.11 has the measurements from before they were hidden) |
1515
| `net_audit.py` | Whether zypi talks to the network (plan C.1). It plays one turn and, every few milliseconds, lists the sockets of zypi and of each process below it (`/proc/<pid>/fd` joined with `/proc/<pid>/net/*`). zypi must have no connection to a non-loopback address, and it must listen on none. `--mock` uses the mock LLM (the agent must be silent too). `--model-from FILE` plays one real turn with the `[model]` table of that `agent.toml`: the agent must then hold an established connection to a non-loopback address (the positive control, which shows that the sampler can see traffic), and the output names no host, address or key. `--no-network` (with `--mock`) runs everything in an empty network namespace (`bwrap --unshare-net`): the turn must still end |
1616
| `status_line_probe.py` | Whether the status row is drawn (`[ui.status_line]`, `type = "command"` and `"builtin"`). With the plain Python agent the row must not be drawn, and a command script must not run. With `status_proxy_agent.py`, which adds the `session_status` message that the old Rust agent sent, the row must be drawn: this second run is the control that shows the config text is right. Execution log section 10.14 has the result |

‎scripts/tui_pty/resume_matrix.py‎

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
GAP and do not fail the run; ``--strict`` makes them fail. The change that lands ADR1 (the pager
1414
asks the agent over ACP instead of reading files) has to close them and drop the ``gap`` text, and
1515
a gap that has closed shows as FIXED until it is dropped. Exit status 0 when nothing FAILed.
16+
``zypi export`` is already on that path (it starts an agent and replays the session with
17+
``session/load``), so its two cases have no gap.
1618
1719
Linux only (it needs a pty); everything but ``run`` is plain Python and is tested on any platform.
1820
"""
@@ -44,6 +46,8 @@ class Outcome:
4446
text: str # the screen, or what a run that ended printed
4547
stdout: str = "" # `zypi export` only
4648
new_sessions: dict[str, str] = field(default_factory=dict) # file name -> id in its header
49+
# `zypi export` only: the processes of the run that were still alive when it ended
50+
left_behind: list[str] = field(default_factory=list)
4751

4852
@property
4953
def replayed(self) -> bool:
@@ -156,17 +160,24 @@ class Case:
156160
"export <id> prints the transcript",
157161
["export", "{existing}"],
158162
"work",
159-
"exit 0 and the conversation on stdout",
160-
lambda o, ids: o.code == 0 and FIRST in o.stdout,
161-
gap="export reads the old layout: \"Session '<id>' not found.\", exit 1",
163+
"exit 0, both sides of the conversation on stdout, no agent left running",
164+
lambda o, ids: (
165+
o.code == 0 and FIRST in o.stdout and REPLY in o.stdout and not o.left_behind
166+
),
162167
export=True,
163168
),
164169
Case(
165170
"export <unknown id> fails without output",
166171
["export", "{unknown}"],
167172
"work",
168-
"exit non-zero, nothing on stdout",
169-
lambda o, ids: o.failed and not o.stdout.strip(),
173+
"exit non-zero, an error that names the id, nothing on stdout, no agent left running",
174+
lambda o, ids: (
175+
o.failed
176+
and not o.stdout.strip()
177+
and ids["unknown"] in o.text
178+
and "not found" in o.text
179+
and not o.left_behind
180+
),
170181
export=True,
171182
),
172183
]
@@ -258,7 +269,13 @@ def finish(term: pt.Term) -> None:
258269
done = subprocess.run(
259270
[zypi, *argv], env=run_env, cwd=cwd, capture_output=True, text=True, timeout=60
260271
)
261-
outcome = Outcome(done.returncode, done.stderr, stdout=done.stdout)
272+
outcome = Outcome(
273+
done.returncode,
274+
done.stderr,
275+
stdout=done.stdout,
276+
# The export starts an agent of its own; it has to stop it before it exits.
277+
left_behind=[cmd for _pid, _state, cmd in pt.processes_with_env(marker)],
278+
)
262279
else:
263280
term = start(argv, cwd)
264281
try:

‎tui/crates/codegen/pi-pager-bin/src/main.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -631,7 +631,7 @@ async fn async_main(args: PagerArgs) -> Result<()> {
631631
}
632632
Command::Export(export_args) => {
633633
init_tracing_simple("cli");
634-
return pi_pager::export_cmd::run(export_args);
634+
return pi_pager::export_cmd::run(export_args).await;
635635
}
636636
Command::Wrap(ref wrap_args) => {
637637
return pi_pager::wrap_cmd::run(wrap_args);

‎tui/crates/codegen/pi-pager/Cargo.toml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,8 @@ criterion = { workspace = true }
170170
tempfile = { workspace = true }
171171
serial_test = { workspace = true }
172172
pi-agent = { workspace = true }
173+
# `#[tokio::test(start_paused = true)]` in the tests of the export's replay reader.
174+
tokio = { workspace = true, features = ["test-util"] }
173175

174176
[[bench]]
175177
name = "render"

‎tui/crates/codegen/pi-pager/docs/user-guide/17-sessions.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,7 @@ zypi export <session-id> notes.md
4242
```
4343

4444
If you omit the file name, `zypi` writes the Markdown to stdout. To copy the Markdown to the clipboard, add `-c`.
45+
46+
`zypi export` asks the agent for the session. It starts the agent, loads the session and stops the agent when it has the text. This takes about as long as `zypi --resume` takes to show the conversation. The Markdown has what a resumed session shows: your messages, the replies of the agent, and one line for each tool call. If the agent compacted the session, the text starts at the first message that the compaction kept.
47+
48+
If no session has the id, `zypi` shows an error and writes nothing.

0 commit comments

Comments
 (0)