Skip to content

test(tui): probes that show what flags and settings do to the Python agent - #57

Merged
zy1233 merged 1 commit into
docs/plan-r24from
test/tui-pty-settings-flags-probes
Oct 31, 2026
Merged

zy1233 merged 1 commit into
docs/plan-r24from
test/tui-pty-settings-flags-probes

Conversation

@zy1233

@zy1233 zy1233 commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Summary

Two probes that answer plan decision 8 (command-line flags) and decision 14 (settings page) with measurements, not reading. No product code changes.

zypi gives the Python agent no argument and no variable but PI_HOME. What a flag or a setting can change for the agent is therefore on the ACP wire or nowhere. Each probe runs zypi twice (without and with the flag or setting), with a logging proxy between zypi and the agent, and prints what is different.

zypi ──ACP──> wire_log_agent.py ──ACP──> context_spy_agent.py (Python agent)
                  │ wire.jsonl                │ spy.jsonl: model, reasoning, tools, system prompt
                  └──────────── wire_probe.summarize / changes(base, run) ────────────┘
scripts/tui_pty/
├── flags_probe.py          # 63 cases: 62 flags (hidden ones that touch the wire or a file too) + a repeat of the base
├── settings_probe.py       # the 37 rows, the permission question, the cursor row, [models].default
├── wire_probe.py           # shared: execute() runs one zypi; summarize() / changes() compare runs
├── wire_log_agent.py       # stand-in agent: copies ACP both ways into a log (stdlib only)
├── context_spy_agent.py    # the Python agent with each LLM call recorded + a scripted tool call
└── README.md               # two table rows, two run lines
scripts/tests/test_tui_probes.py   # 16 tests: the helpers and the stand-in agent (no zypi, no pty)

Evidence

Run against the v0.7.0 Linux binary (7cb9e23, WSL2), mock LLM, throw-away home. The base run repeats itself ((no flag, second run): NO CHANGE).

Flags (full run: 63 cases, 282 s). Not one of these changes the LLM call:

Result Flags
NO CHANGE --allow --deny --allowedTools (a deny rule: the question is still asked), --reasoning-effort, --tools, --disallowed-tools, --disable-web-search, --agents, --max-turns, --verbatim, --output-format, --include-partial-messages, --json-schema, --oauth, --trust, --force-login
Only _meta on the wire, which the agent does not read --rules / --append-system-prompt, --system-prompt-override / --system-prompt (the system prompt of the LLM call is the same), --no-plan, --no-subagents, --no-ask-user, --agent
Zypi says it cannot, and goes on --worktree, --worktree-ref: the toast "Git worktree sessions are not supported in standard ACP mode"; the session starts in the main checkout
Works --cwd, --debug / --debug-file, --no-alt-screen, --minimal, positional prompt, the always-approve family, --permission-mode, -m (a configured id), --session-id

Settings (37 rows on the page):

remember_tool_approvals   unset | true | false : 2 questions each, answers: [Allow once, No, reject]
                          -> no "Always allow ..." row exists; the setting has nothing to switch
default_selected_permission   unset / always_allow_all_sessions / allow_once / allow_command_always
                          -> cursor on "Allow once"   (the page says "Always allow on all sessions")
                          reject -> cursor on "No, reject"   (the only value that does something)
[models] default = mock-b label after the turn: mock-a; LLM call used: mock/mock-a
                          -> the "Default model" row switches this session; the next start ignores it
hunk_tracker_mode, fork_secondary_model, prompt_suggestions, remember_tool_approvals,
[cli] auto_update, [toolset.ask_user_question] timeout_enabled, all set at once
                          -> NO CHANGE on the wire, in the agent's variables, in the LLM call
tools the LLM is offered  bash edit fetch_url find goal_complete goal_update grep ls read web_search workflow write
                          -> no ask_user_question tool
  • ruff check . and ruff format --check . pass. scripts/tests/test_tui_probes.py: 16 passed on Windows and on Linux (WSL2).
  • The probes judge nothing; they print. The next PR reads the output and acts on it.

Merge Danger

Door: two-way.

Blast Radius: scripts and one test file; nothing in pi_agent_core, the packages or tui/.

…agent

flags_probe.py runs zypi once per command-line flag and lists what changes on the agent's side of the ACP wire (plan 1.P6, decision 8). settings_probe.py does the same for the settings page (decision 14): the rows, the permission question under remember_tool_approvals, the cursor row per default_selected_permission, [models].default, and the settings meant for the old runtime. wire_log_agent.py and context_spy_agent.py are the stand-in agents the probes use. No product code changes.
@zy1233

zy1233 commented Oct 31, 2026

Copy link
Copy Markdown
Owner Author

Stack #66, bottom to top: #58, #59, #56, #57, #60, #61, #62, #63, #64, #65. One atomic squash merge (gh stack merge).

Position 4 of 10. Base: docs/plan-r24 (#56). Head a526a28 -> 5152753. The patch is the same except one docstring: scripts/tui_pty/settings_probe.py now points to plan section 10.30 (it said 10.27, the number of an earlier version of the round record).

@zy1233
zy1233 merged commit fb4cc65 into main Oct 31, 2026
4 checks passed
@zy1233
zy1233 deleted the test/tui-pty-settings-flags-probes branch October 31, 2026 03:17
@zy1233 zy1233 mentioned this pull request Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant