fix(ios): classify the keyboard dismiss refusal by what it dispatched by thymikee · Pull Request #4536 · callstack/agent-device

thymikee · 2026-11-13T23:42:56Z

Summary

keyboard dismiss refused two different verdicts with one untyped message, so a caller had to match prose to tell them apart: a keyboard that exposes no dismiss control (nothing was sent) and a dismiss control that was tapped and left the keys up (it dispatched). The refusal behavior is unchanged; only its classification becomes typed.

The runner answers the no-dismiss-key case with its own wire code KEYBOARD_DISMISS_UNAVAILABLE, classified by the existing runner-code table. The published code contract is unchanged — it still travels under UNSUPPORTED_OPERATION through its own wire-code map — and the failure now carries details.reason: keyboard_dismiss_unavailable, details.runnerErrorCode, and dispatched: no, all visible in --json. dismissKeyboard answers a RunnerKeyboardDismissOutcome carrying dispatchedDismissal, which also decides which verdict a tvOS keyboard stayed up for.

14 files: Swift runner + protocol doc, the Apple runner classification and its tests, the dispatch-disclosure fixture, command help, and the command docs. No scope expansion beyond keyboard dismiss; no CHANGELOG.

$ agent-device keyboard dismiss --json
{"error":{"code":"UNSUPPORTED_OPERATION","message":"Unable to dismiss the keyboard: it exposes no dismiss key, …",
 "details":{"runnerErrorCode":"KEYBOARD_DISMISS_UNAVAILABLE","reason":"keyboard_dismiss_unavailable","dispatched":"no"}}}

Live simulator evidence

Head 306352ce2a. iPhone 17, iOS 27.0, UDID 0E0FCBDC-88E2-4CB6-A1DA-E148F77743F4, Xcode 27.1 (27A9275). Runner rebuilt from this branch with pnpm build && pnpm build:xcuitest; fresh state dir, so a new daemon.

A="node bin/agent-device.mjs --platform ios --udid 0E0FCBDC-88E2-4CB6-A1DA-E148F77743F4 --state-dir /private/tmp/ad-4536-state --session k4536"

No dismiss control — Agent Device Tester, Form tab, Full name field, software keyboard up with no Done/Hide key:

$A open com.callstack.agentdevicelab --relaunch; $A press @e27; $A press @e10
$A keyboard dismiss --json     # exit 1
{"success":false,"error":{"code":"UNSUPPORTED_OPERATION","message":"Unable to dismiss the keyboard: it exposes no dismiss key, and background taps are never attempted (…)","hint":"An element whose center sits behind the on-screen keyboard is refused with tap_keyboard_occludes_target; … or use keyboard enter to press the return key when submission is wanted.",
 "details":{"runnerErrorCode":"KEYBOARD_DISMISS_UNAVAILABLE","reason":"keyboard_dismiss_unavailable","dispatched":"no"}}}

A screenshot taken right after shows the full QWERTY keyboard still visible, and snapshot -i still lists the key nodes.

With dismiss control — Safari, https://the-internet.herokuapp.com/login, Password field, accessory bar with Previous/Next/Done:

$A open com.apple.mobilesafari https://the-internet.herokuapp.com/login; $A press @e16
$A keyboard dismiss --json
{"success":true,"data":{"platform":"ios","action":"dismiss","wasVisible":true,"dismissed":true,"visible":false,"mechanism":"dismissKey","message":"Keyboard dismissed via its dismiss key"}}

snapshot -i afterwards no longer lists the Previous/Next/Done toolbar.

Cleanup: close --shutdown, daemon stop --clean --state-dir /private/tmp/ad-4536-state.

Validation

Base ddbfc85566, functional head b46ac6d761; 3231d4448c and 579ad97b28 are doc/help-sentence follow-ups on review; 306352ce2a applies the review recommendations (flag rename, separate wire-code map, help sentence).

pnpm check:affected --run --head b46ac6d76170efc6867e22acb773d0f57afd2c14 — all runnable checks passed, exactCommit=true (format, lint, typecheck, layering, fallow, native-dupes, build, vitest-related, unit, command-docs, docs-links, docs-prose, xctest-selection, packaged-runner-swift).
Review follow-ups 3231d4448c and 579ad97b28 touch only the keyboard help sentence and its docs counterpart. For those: cli-help-command-usage.test.ts, cli-help-topics.test.ts, command-doc-coverage.test.ts, check:doc-prose, and the MCP-metadata check pass locally; the serialized gate for the head is owned by CI.
pnpm build:xcuitest (iOS + macOS) succeeded; pnpm check:xctest-selection reports the new Swift test reachable from the host lane.
Re-run at 306352ce2a: focused vitest (runner-error-classification, runner-contract-response, index-dispatch-disclosure, src/commands/schema, help-conformance), pnpm check:xctest-selection, pnpm build && pnpm build:xcuitest pass.
CI obligations: device lanes (PR iOS runner smoke, nightly XCTest) run on GitHub. Coverage passed at 579ad97.
Unresolved risk: no released tag contains the old details shape, and new optional keys inside the open details record are already declared additive in the wire-compat ledger, so no ledger entry was added.

🤖 Generated with Claude Code

`keyboard dismiss` refused two different things with one untyped message, so a caller had to match prose to tell them apart. A keyboard that exposes no dismiss key and a dismiss control that was tapped and left the keys up are different verdicts: the first sent nothing, the second dispatched. The runner now answers the no-dismiss-key refusal with its own wire code, KEYBOARD_DISMISS_UNAVAILABLE, declared in RunnerWireErrorCode and classified by the existing runner-code table. It keeps the code the command already refused with, UNSUPPORTED_OPERATION, through a wireCode row, so the published code contract is unchanged, and it gains details.reason keyboard_dismiss_unavailable with dispatched: no. dismissKeyboard answers a RunnerKeyboardDismissOutcome carrying dismissControlFound, which also decides which of the two verdicts a tvOS keyboard stayed up for. Help, the runner protocol, and the command docs state which refusal sends nothing. Co-Authored-By: Claude Opus 5.5 <[email protected]>

github-actions · 2026-11-13T23:44:04Z

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	6.27 MB	6.27 MB	+1.4 kB
Package (unpacked)	6.27 MB	6.27 MB	+1.4 kB
Package (download)	1.90 MB	1.90 MB	+401 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	24.8 ms	27.3 ms	+2.5 ms
CLI --help	50.2 ms	52.5 ms	+2.4 ms

cubic-dev-ai

All reported issues were addressed across 14 files

_{Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic}

Co-Authored-By: Claude Opus 5.5 <[email protected]>

thymikee · 2026-11-13T23:53:54Z

I reviewed b46ac6d. The change has no live device run yet, and it rewrites the iOS keyboard dismiss dispatch path, so it needs one before merge.

The PR changes the return type of dismissKeyboard and its three outcome sites (tvOS Menu press, iOS dismiss-key tap, no-control refusal) in RunnerTests+Keyboard.swift. It also changes the keyboardDismiss case in RunnerTests+CommandExecution.swift. This is the device-facing dispatch path, and the PR body says it was not verified on a live simulator. The Swift test calls keyboardDismissRefusal(dismissControlFound:) directly, and the TS tests use hand-written runner replies. Nothing shows the real route where a software keyboard with no hittable Done/Hide control reaches the refusal with dismissControlFound == false. If that flag is wrong on the real route, dispatched: no (which tells callers a resend is safe) could be published for a command that did tap. The refusal could also keep the generic code, so the new classification would never fire. Please run agent-device keyboard dismiss --json on an iOS simulator against a field whose software keyboard has no Done/Hide control, such as Agent Device Tester. The output must show error.code UNSUPPORTED_OPERATION, error.details.runnerErrorCode KEYBOARD_DISMISS_UNAVAILABLE, reason keyboard_dismiss_unavailable, dispatched no and the hint. The keyboard must stay visible. Then run it on a keyboard with a Done/Hide control and show the success path is unchanged (dismissed: true, mechanism dismissKey). Please record the commands, the device and the head sha. The Smoke Tests (ios) job counts only if its log shows these commands.

The two inline threads on the help text in packages/command-surface/src/system/index.ts (Android refusal, tapped-and-stayed-up refusal) applied at b46ac6d; the newer head 579ad97 rewords that help and commands.md, which looks like it addresses both, and the next pass will confirm.

Not blocking, and you can take or leave these: DIAGNOSTIC_ONLY_RUNNER_ERROR_CODES in runner-contract.ts now holds a row that renames its code on the wire, so a separate name or map would describe it better. dismissControlFound in RunnerTests+Keyboard.swift means a dismissal was dispatched (on tvOS it is the Menu press), so a name like dispatchedDismissal would match. The keyboard paragraph in cli-help.ts does not mention error.details.reason or dispatched yet.

I read the code only and did not run xcodebuild, vitest or the CLI. The check:affected, build:xcuitest and check:xctest-selection results are as the author reported them.

Coverage failed, and I could not tell from the log whether this diff causes it. The diff touches code that the coverage run executes, so please check that job. Smoke Tests, Smoke Tests (ios) and Smoke Tests (macos) were still running. The iOS and macOS lanes build the runner this PR changes, so a failure there would relate to it. No conflicts were reported.

Before merge, please add the live simulator evidence above and get Smoke Tests (ios and macos) and Coverage green at the head.

apex-sentinel-staging · 2026-11-13T23:55:39Z

Apex Sentinel · ✅ Approve

Since the last review: 0 new · 0 still open · 1 no longer raised

No open findings.

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (ddbfc855661f).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command	What it does
`@apex-sentinel-staging note <context>`	Adds context; Sentinel re-checks the cited code
`@apex-sentinel-staging pivot <why>`	Says the finding is wrong; Sentinel re-checks the cited code
`@apex-sentinel-staging like`	Marks the finding as useful
`@apex-sentinel-staging dislike <why>`	Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

Earlier reviews

Review of c742a001 · 14.11.2026 00:12 UTC

Apex Sentinel · ✅ Approve

Since the last review: 1 new · 0 still open · 0 no longer raised

1 finding(s): 1 minor.

🟡 Minor (1)

Where	Finding
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Keyboard.swift:222`	🆕 tvOS dispatchedDismissal contradicts its own comment and can report the nothing-was-sent code

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (ddbfc855661f).

Review of 579ad97b · 13.11.2026 23:55 UTC

Apex Sentinel · ✅ Approve

No open findings.

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (ddbfc855661f).

…code map Rename the dismiss outcome flag to dispatchedDismissal, since it also covers the tvOS Menu press. Move the keyboard-dismiss wire-code rename out of the diagnostic-only list into its own map, and name the iOS reason in the workflow help. Co-Authored-By: Claude Opus 5.5 <[email protected]>

thymikee · 2026-11-14T00:00:27Z

Thanks for the review. I pushed 306352ce2a and ran the live simulator check.

Recommendations applied:

dismissControlFound is now dispatchedDismissal in the Swift runner, the command execution call site, and the Swift test.
The keyboard-dismiss wire rename now lives in its own map, DIAGNOSTIC_RUNNER_ERROR_WIRE_CODES, in runner-contract.ts. DIAGNOSTIC_ONLY_RUNNER_ERROR_CODES only classifies again. Behavior is the same.
The workflow help in cli-help.ts now names error.details.reason keyboard_dismiss_unavailable (iOS) and dispatched: no. That topic has a size budget, so I shortened two nearby phrases to stay under it.

Live evidence at 306352ce2a, on an iPhone 17 (iOS 27.0, Xcode 27.1):

No dismiss key (Agent Device Tester, Full name field, keyboard up): UNSUPPORTED_OPERATION, runnerErrorCode KEYBOARD_DISMISS_UNAVAILABLE, reason keyboard_dismiss_unavailable, dispatched: no, and the hint is present. A screenshot after the command still shows the keyboard.
With a dismiss key (Safari login Password field, Previous/Next/Done bar): success: true, dismissed: true, mechanism: dismissKey. The toolbar is gone afterwards.

The commands and the JSON are in the new "Live simulator evidence" section of the PR body.

Coverage passed on 579ad97. The earlier failure in daemon-client-ensuring.test.ts was an unrelated flake. This PR touches no daemon-client files.

The new head needs its own CI run, including Smoke Tests (ios and macos).

cubic-dev-ai

All reported issues were addressed across 5 files (changes from recent commits).

_{Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic}

Co-Authored-By: Claude Opus 5.5 <[email protected]>

cubic-dev-ai

All reported issues were addressed across 1 file (changes from recent commits).

_{Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic}

apex-sentinel-staging

Apex Sentinel: 1 new finding on c742a00. The verdict and the full list are in the summary comment.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

thymikee · 2026-11-14T00:21:59Z

Final head is a0c5aec798. All 22 checks pass, including Smoke Tests (ios), Smoke Tests (macos) and Coverage. There are no open review threads.

The live simulator evidence in the PR body was recorded at 306352ce2a. The two later commits don't change the iOS dismiss path:

c742a0016a restores "Element presses" in the help text.
a0c5aec798 passes dispatchedDismissal: true in the tvOS branch only, because pressTvRemote(.menu) always returns true on tvOS.

thymikee · 2026-11-14T00:53:12Z

This PR is ready at a0c5aec. I found no problems in the code, and all 22 checks pass, including the iOS and macOS smoke tests that build the runner this diff changes and the coverage check. Not blocking: the only Swift test calls keyboardDismissRefusal(dispatchedDismissal:) directly, so no test or live run covers a dismiss key that is tapped and leaves the keyboard visible (hard to reproduce, so optional), and the help edit also rewords two unrelated phrases in cli-help.ts ("also Limrun iOS" and "never tap static text/headings") to fit the size cap, so a line in the PR body would help; take or leave both.

The live runs in the PR body were recorded at 306352c, not the final head. From the diff, the two later commits touch only cli-help.ts and the tvOS-only branch. I did not re-run anything, and no live run covers tvOS. Those codes and details come from the classification test and a code read.

Of the earlier review threads, none still applies. These are fixed at this head, so please resolve them: #4536 (comment) (help wording, both platforms), #4536 (comment) (tapped-key outcome), #4536 (comment) (keyboard occlusion wording) and #4536 (comment) (tvOS dispatch flag). This one is benign, because nothing in the line is false and the command help lists both outcomes: #4536 (comment). Nothing else must happen before merge.

* origin/main: fix(android): carry an empty text on a node labelled by its content description (#4531) fix(ios): restore the back swipe's pre-swipe still span (#4515) fix(daemon-client): type the remote daemon refusal with its dispatch fact (#4532) fix: prepare reinstall artifacts before removing apps (#4533) fix(ios): classify the keyboard dismiss refusal by what it dispatched (#4536) fix(ios): press the navigation bar back button and speed up the back swipe (#4508) refactor(gestures): one owner for the densified gesture path; drop the unproven sendevent transport (#4239) fix(host-kit): keep Linux process identity stable across clock steps (#4438) fix(ios): bound the idle wait of keyboard dismiss and keyboard enter (#4454) fix(device-selection): fail an ambiguous device request with only the tied devices (#4506)

cubic-dev-ai Bot reviewed Nov 13, 2026

View reviewed changes

Comment thread packages/command-surface/src/system/index.ts Resolved

Comment thread packages/command-surface/src/system/index.ts Resolved

thymikee and others added 2 commits November 14, 2026 00:47

docs(keyboard): name both dismiss outcomes on both platforms

3231d44

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(keyboard): state the shared refusal before the iOS-only reason

579ad97

Co-Authored-By: Claude Opus 5.5 <[email protected]>

cubic-dev-ai Bot reviewed Nov 14, 2026

View reviewed changes

Comment thread src/commands/schema/cli-help.ts Resolved

docs(help): scope the keyboard occlusion refusal to element presses

c742a00

Co-Authored-By: Claude Opus 5.5 <[email protected]>

cubic-dev-ai Bot reviewed Nov 14, 2026

View reviewed changes

Comment thread src/commands/schema/cli-help.ts Resolved

apex-sentinel-staging Bot reviewed Nov 14, 2026

View reviewed changes

Comment thread apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Keyboard.swift Resolved

fix(ios): pass the tvOS Menu press as a dispatched dismissal

a0c5aec

Co-Authored-By: Claude Opus 5.5 <[email protected]>

thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 14, 2026

thymikee merged commit 13cfb23 into main Nov 14, 2026
22 checks passed

thymikee deleted the fix/ios-keyboard-dismiss-reason branch November 14, 2026 07:26

Conversation

thymikee commented Nov 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Live simulator evidence

Validation

Uh oh!

github-actions Bot commented Nov 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Report

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thymikee commented Nov 13, 2026

Uh oh!

apex-sentinel-staging Bot commented Nov 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Apex Sentinel · ✅ Approve

Earlier reviews

Apex Sentinel · ✅ Approve

🟡 Minor (1)

Apex Sentinel · ✅ Approve

Uh oh!

thymikee commented Nov 14, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

apex-sentinel-staging Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

thymikee commented Nov 14, 2026

Uh oh!

thymikee commented Nov 14, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thymikee commented Nov 13, 2026 •

edited

Loading

github-actions Bot commented Nov 13, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

apex-sentinel-staging Bot commented Nov 13, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading