Repository navigation
Commit ddbfc85
fix(ios): press the navigation bar back button and speed up the back swipe (#4508)
* fix(ios): press the navigation bar back button and speed up the back swipe
iOS back now reads the current tree in the same runner call for the navigation bar's system back
button (identifier BackButton inside a navigation bar). When exactly one is hittable, it taps it with
a synthesized tap and the daemon's settle observes the pop. No other button counts, whatever its
label; an unreadable or penalized tree, or two candidates, fall back to the swipe.
The swipe is resolved before settling, samples 0.05 s apart, and measures the change from the last
still sample, so the separate just-before capture is gone. The post-swipe delay drops from 0.25 s to
0.1 s, which also shortens the required still span. The settle budget is 2 s. The gap-coverage and
baseline checks still decide every success.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): confirm the back button tap and scale the settle budget
The navigation bar back button path now reads the bars again after the tap (one
XCTest snapshot per bar, 0.05 s apart, up to 1 s) and answers
BACK_SWIPE_UNVERIFIED when neither the bar title nor its back button changed; a
delivered tap is never followed by the swipe. The button log line is emitted on
every exit. The back swipe settle budget grows with the slowest capture so a
still screen can settle on a slow host. Help, flag text, command docs, and the
runner protocol describe when the button is pressed and what each fallback
answers.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* docs(back): scope the iOS swipe fallback and state the settle budget plainly
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): hold XCTest issues in the back button reads
Every navigation bar read in iOS back runs inside holdingRecordedIssues.
A held issue before the tap means no button, so back swipes. A held issue
after the tap proves nothing, so back answers back_swipe_unverified and
never swipes. This replaces the suppressed-issue counter diff, which only
saw kAXError issues.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): hold the back bar still before the tap and surface a hung query
- Read the bar state twice around the button lookup and tap only when the
two reads agree and contain the button's frame, so a push still
animating is not taken for the state a pop changes.
- Compare bar count, identifier, static-text titles, and back buttons.
- Start bar reads only within the 1 s budget, checked before each bar's
snapshot; document what bounds a read in flight.
- Re-record a hung XCTest query as record(_:) requires, send nothing
more: BACK_SWIPE_NOT_DISPATCHED before the tap, BACK_SWIPE_UNVERIFIED
after it.
- Drop the dead outcome guard on the XCTest tap fallback.
- Correct the visionOS back hints; a thrown tap no longer logs a swipe
verification error line.
- Test the two-candidate rule, the skipped read, and hung queries; the
lookup-issue test asserts the swipe path, not a swipe outcome.
- Name the simulator still span in commands.md.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): match the back button in screen space on a placed iPad window
A live XCUIElement frame is in screen points, but a snapshot frame of a
placed iPad window starts at the window's own top-left (#3899). The
pre-tap still check compared the two directly, so on a placed window the
button was never found still: back spent the 1 s read budget and swiped.
The check now places the snapshot frames at the window's screen origin
through the synthesized coordinate context's SurfaceGeometry.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* test(ios): keep the placed-window back-button test iOS-only
The macOS runner test build has no navigationBarBackState.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): confirm back only from the back button and space the still reads
The still check before the tap now needs two agreeing bar reads started at least one read
interval apart, so two reads drawn in one animation frame no longer pass for a still bar.
The bar state is now each bar's back button label and frame, so a bar title that updates
itself, such as a countdown, can no longer confirm a tap that navigated nowhere. The
placed-window button match allows 0.5 pt per edge for fractional frame values.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): confirm a back button pop only from bar and back button labels and counts
A tap that navigates nowhere can still move the back button, such as by a
large-title collapse, so its frame no longer confirms a pop. Frames stay in
the still check before the tap and in matching the button to press.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* refactor(ios): drop the timed read and frame tolerance from the back button check
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(ios): read the bar once more after the back button confirmation budget
A snapshot of a navigation bar that vanished mid-pop spends about 2 s in
XCTest's element-resolution retries before it raises a held issue. On a
slow CI simulator that read used up the 1 s confirmation budget, so no
read saw the settled screen and back answered back_swipe_unverified
after a real pop. One read now always follows the budget, with no
deadline, so the screen after the pop is read.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* test(ios): fail every in-budget back confirmation read, not a fixed count
The slow-host confirmation test failed two reads that each slept 0.6 s,
so whether the final read came third depended on real read speed. The
confirmation loop now tells the test hook which read starts after the
budget, so the test fails every read in the budget and succeeds only on
that final read. The protocol and command docs now say the final read
can also fail or hang, and that a read can run well past the 1 s.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* docs(back): name the final confirmation read in back_swipe_unverified
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* docs(back): describe NavigationBarBackReadPhase as the read hook's phase
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent b68ed4b commit ddbfc85
11 files changed
Lines changed: 949 additions & 104 deletions
File tree
- apple/runner
- AgentDeviceRunner
- AgentDeviceRunnerUITests
- UnitTests
- AgentDeviceRunner
- packages
- command-registry/src
- command-surface/src/system
- platform-apple/src
- website/docs/docs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
42 | 153 | | |
43 | 154 | | |
44 | 155 | | |
| |||
422 | 533 | | |
423 | 534 | | |
424 | 535 | | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
425 | 547 | | |
426 | 548 | | |
427 | 549 | | |
| |||
662 | 784 | | |
663 | 785 | | |
664 | 786 | | |
665 | | - | |
| 787 | + | |
666 | 788 | | |
667 | 789 | | |
668 | 790 | | |
669 | 791 | | |
670 | | - | |
| 792 | + | |
671 | 793 | | |
672 | 794 | | |
673 | 795 | | |
| |||
0 commit comments