Repository navigation
Commit fa8cce3
authored
refactor(ios): snapshot capture plans with a structured quality verdict (#783)
* refactor(ios): snapshot capture plans with a structured quality verdict
Implements ADR 0004's explicit-strategies decision as architecture
(candidates 1+2 of the snapshot pipeline review):
- Snapshot backend seam: three adapters (recursive tree, query sweep,
private AX) behind one captureWithBackend dispatch. Each strategy
declares its chain as data (regular: tree→queries→private-ax,
compact: queries→private-ax, raw: tree→private-ax) and one plan
runner walks it under a 20s umbrella budget so chained recovery can
never stack past the 30s main-thread watchdog. Terminal policy is
per-plan: raw rethrows AX failures (diagnostics preserve errors),
interactive fails closed with runnerFatal invalidation.
- Single quality classifier: one sparsePayloadReason predicate (with
reason codes), one collapsed-leaf detector, replacing the three
divergent sparse detectors (Swift structural, daemon count==1, CLI
count<=3) that each patched a different failure shape.
- Structured snapshot quality verdict on the wire (state, backend,
reason, reasonCode, effectiveDepth, collapsedLeafIndexes): the daemon
and CLI render warnings from it instead of re-deriving degradation
from node shapes; budget starvation is no longer blamed on the app's
accessibility. Legacy runner messages and daemon-side detectors stay
behind a verdict-absent gate for mixed-version compat.
- The verdict surfaces in --json (snapshotQuality) for agents; the
generic sparse CLI hint is suppressed when a verdict explains it.
Threading the verdict exposed two more hand-copy field drops
(captureInteractionOutcomeAwareSnapshot, serializeSnapshotResult,
client response mapping) - now carried alongside warnings everywhere.
Verified live: Settings healthy (tree, no warnings), Settings compact
under load (recovered/private-ax/budget), production login (sparse
best-effort with honest warning), collapse fixture (healthy +
collapsedLeafIndexes -> @ref warning), Bluesky Home (recovered/
private-ax, 24 nodes in 2s). Full unit suite 2327 passed, fallow clean,
runner builds.
* fix(ios): correct recovered-snapshot viewport and private-AX scope semantics
Review follow-ups on the capture-plan refactor:
- The query-sweep synthetic root doubles as the daemon's viewport
(find.ts prefers on-screen matches inside nodes[0].rect), but it was
built from candidate bounds, so off-screen controls below the screen
could inflate it and win duplicate-label resolution. The root now
uses the real finite viewport, falling back to candidate bounds only
when viewport capture failed.
- The private-AX backend applied --scope as a per-node text filter,
hiding the matched container's children — diverging from regular
snapshot scope semantics and contradicting the depth-cap hint that
recommends scoped re-runs. Scope now selects the matched subtree:
descendants inherit the match and only the normal option filters
apply to them (in-bundle test covers a non-matching descendant).
Verified live on Bluesky Home: scope homeScreen returns the 52-node
subtree including non-matching descendants; compact root rect equals
the screen (0,0,402,874).
* fix(ios): fail closed on interactive AX failure, stamp fatal verdict, validate parser
Three review findings on the capture-plan terminal path:
- P1: the fail-closed guard required `best == nil`, but the query-sweep
tier always returns a synthetic-root sparse payload that sets `best` —
so an interactive recursive-tree AX serialization failure that no
backend recovered returned a sparse snapshot instead of invalidating
the cached target. Reaching the terminal already means no backend
produced a usable tree, so the sparse `best` must not suppress the
fail-closed path. Extracted the decision into a pure, unit-tested
`resolveSnapshotPlanTerminal` (closes the terminal-ordering testability
gap the architecture review flagged).
- P2: `snapshotAccessibilityUnavailable` returned a payload with no
`snapshotQuality`, leaving one planned sparse result on the
legacy-message path. It now carries a sparse/ax-rejected verdict like
every other planned snapshot, so downstream sparse handling keys off
the verdict.
- P2: `readSnapshotQualityVerdict` cast any string state/backend into the
union, so a malformed object suppressed the legacy node-shape
detectors. State and backend are now validated against their unions
(unknown → verdict-absent → legacy detectors run); an unknown
reasonCode is dropped rather than rejecting the whole verdict, so a
forward-version runner still yields a usable verdict.
Unit-covered: Swift resolveSnapshotPlanTerminal matrix + fatal-verdict
assertion; TS parser accept/reject/forward-compat. Full suite 249 files
/ 2449 tests, fallow, lint, runner build green.1 parent 3b94b36 commit fa8cce3
18 files changed
Lines changed: 819 additions & 444 deletions
File tree
- docs/adr
- ios-runner/AgentDeviceRunner/AgentDeviceRunnerUITests
- src
- __tests__
- commands
- core/interactors
- daemon
- handlers
- utils
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
6 | 8 | | |
7 | 9 | | |
8 | 10 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | | - | |
16 | | - | |
17 | | - | |
| 15 | + | |
| 16 | + | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
| |||
66 | 65 | | |
67 | 66 | | |
68 | 67 | | |
69 | | - | |
| 68 | + | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | 77 | | |
85 | | - | |
86 | | - | |
| 78 | + | |
87 | 79 | | |
88 | | - | |
89 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
90 | 85 | | |
91 | | - | |
92 | 86 | | |
93 | 87 | | |
94 | 88 | | |
| |||
100 | 94 | | |
101 | 95 | | |
102 | 96 | | |
103 | | - | |
| 97 | + | |
| 98 | + | |
104 | 99 | | |
105 | 100 | | |
106 | 101 | | |
| |||
115 | 110 | | |
116 | 111 | | |
117 | 112 | | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
118 | 126 | | |
119 | 127 | | |
120 | 128 | | |
| 129 | + | |
| 130 | + | |
121 | 131 | | |
122 | 132 | | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | 133 | | |
127 | 134 | | |
128 | 135 | | |
| |||
164 | 171 | | |
165 | 172 | | |
166 | 173 | | |
167 | | - | |
| 174 | + | |
| 175 | + | |
168 | 176 | | |
169 | 177 | | |
170 | 178 | | |
| |||
225 | 233 | | |
226 | 234 | | |
227 | 235 | | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
205 | 205 | | |
206 | 206 | | |
207 | 207 | | |
| 208 | + | |
208 | 209 | | |
209 | 210 | | |
210 | 211 | | |
| |||
242 | 243 | | |
243 | 244 | | |
244 | 245 | | |
| 246 | + | |
245 | 247 | | |
246 | 248 | | |
247 | 249 | | |
| |||
278 | 280 | | |
279 | 281 | | |
280 | 282 | | |
| 283 | + | |
281 | 284 | | |
282 | 285 | | |
283 | 286 | | |
| |||
0 commit comments