feat(cua-driver): add display_only to get_window_state for live previews - #4881

Why

T3 Code embeds Cua Driver and shows the window the agent drives in a picture-in-picture preview. Its PR pingdotgg/t3code#16975 polls get_window_state every 250 ms. On macOS and Windows each poll replaced or re-owned the agent's snapshot for that window:

• Windows published an empty snapshot, which staled every element token.
• macOS moved the screenshot frame to the preview's session.

Either way, the agent's tokens or pixel actions failed between its own reads, which looks like a flaky driver ("stale token", "no screenshot owned by this session"). T3 fixed this only for Linux, in its own patch of our 0.34.0 source.

What

• display_only:true on get_window_state (requires include_accessibility_tree:false). It returns the window's pixels and metadata, but publishes, refreshes and removes no snapshot, and registers no action capture. It maps onto the existing internal observation-only read on macOS, Windows and Linux. On macOS it also skips the background_input probe, since the frame cannot ground input.
• Same contract as T3's Linux patch, so T3 picks it up with no change on its side:
  • T3 detects the feature by "display_only" in input_schema.properties. The property is added in the shared window_state_view::extend_input_schema, so every backend advertises it.
  • The schema text and the refusal display_only requires include_accessibility_tree:false match T3's patch word for word.
  • The response carries display_only:true and the same frame_note, plus the usual screenshot_width and screenshot_height, which T3 requires.
• Plain screenshot-only reads on Windows and Linux now refresh the current snapshot's frame in place (refresh_screenshot_for_session, as macOS already does). They no longer publish an empty snapshot that invalidated every token. A window with no element snapshot still gets one, as before.
• Skill text: drops "macOS only" from the screenshot-only note, and mentions display_only in WORKFLOW.md.

Not changed: a non-display-only screenshot read from another session still takes over the screenshot frame. That is the same session rule as before, and display_only is the way for a viewer to avoid it.

Tests

• Core: the schema advertises the property, the refusal is exact, and the response is marked.
• Per platform: T3's display_only_refuses_accessibility_snapshot_requests, plus a schema check (macOS run locally; Windows and Linux in CI).
• The live smoke (ci(cua-driver): live smoke of the agent loop on Windows and Linux #4769) has a new check, tokens stay valid after 8 display_only reads, on Windows (WinForms) and Linux (GTK3/Xvfb). It also covers:
  • tools/list advertises the property;
  • each preview read carries display_only/frame_note/dimensions and no snapshot_id/capture_id/invalidated_snapshot_ids;
  • a tree walk with display_only is refused;
  • a plain screenshot-only read invalidates nothing;
  • a click with the token from before the previews still lands.
• macOS: a live check in a Lume VM will be added as a comment when the VM slot frees up.

Credit

The display_only contract is ported from T3 Code's Linux patch to Cua Driver 0.34.0 (pingdotgg/t3code#16975, patches/cua-driver-linux/0.34.0.patch). T3 Code is MIT-licensed, Copyright (c) 2026 T3 Tools Inc. The schema text, the refusal and the result fields match theirs. The THIRD_PARTY_NOTICES row comes with #4877. This supersedes #4878 (Linux only).

Linear: CUA-1232

🤖 Generated with Claude Code