Skip to content

Commit d7a1f2c

Browse files
authored
refactor(ios-runner): make the snapshot quality verdict state a closed enum (#2888)
* refactor(ios-runner): close the snapshot quality verdict state vocabulary The runner wrote the verdict `state` as a free String while the host accepts only healthy | recovered | sparse, and the contracts annotation reader cast any string state into the verdict type. A typo or a one-sided rename therefore dropped the verdict and its disclosure. `SnapshotQualityState` now owns the runner side, raw-value Codable keeps the wire JSON unchanged, and `reasonCode` stays open. The kernel states the vocabulary once as `SNAPSHOT_QUALITY_STATES` with `isSnapshotQualityState`; capture-kit and contracts both read through it, so neither keeps a second accepted-state set and the annotation reader rejects an unknown state instead of casting it. `contracts/fixtures/ios-snapshot-quality-states.json` is the table the Swift `allCases` order and the kernel tuple are each pinned to. * refactor(substrate): hold the verdict state map in each eager-frozen reader The readers reached the vocabulary through `isSnapshotQualityState` in `kernel/snapshot.ts`, which made that module eager in six entry closures the eager-closure gate holds at its merge-base size — the contracts capture façade at 9 modules, capture-kit's verdict reader at 2. The only module those two closures already evaluate is the one whose edge the gate rejects, so no single runtime home for the set exists. Each reader now keys a `Record<SnapshotQualityState, true>` over the kernel union, the home the gate prescribes for code that has to live where it is already evaluated. Membership goes through `Object.hasOwn`, so an inherited key is never a state, and a state added to the tuple without a key in a reader is a compile error there: the guarantee the shared import was bought for, without the eager edge. Both reader tests walk the tuple and the kernel test still pins it to the fixture. * refactor(snapshot): gate the backend name, and stop pinning vocabulary order The annotation reader was one predicate plus a blanket cast. `backend` names the recovery strategy in the warning line, so it goes through a declared map keyed against the kernel union now, and capture-kit lost both of its `as` casts on the way: the strategy is gated on the capability table it already imports, the reason code on an exhaustive map over its union. A full projection in contracts was tried and the repo's own gates refuse it — fallow reports a 4-group, 104-line clone family against capture-kit's normalizer, on top of the eager-closure gate that already forbids a shared reader. The re-read stays in the shape `readTargetActivation` in the same file uses: check the two names that decide presentation, forward what this module published, and pin the pair payload-by-payload from capture-kit's test. Order pinning drops out. The fixture is compared as a set on both sides, the stamping test asserts each case's own raw value, and the unread 28-line capture builder and its import are gone. * refactor(contracts): name the verdict re-read apart from the strict reader Two same-named readers with different trust levels is how the half-migration read as validating more than it does. This module's version checks the two load-bearing names and forwards what this repo published, in the shape `readTargetActivation` uses, so it is now `readPublishedSnapshotQualityVerdict` and the strict per-field reading keeps the plain name for capture-kit's untrusted-payload reader. * refactor(snapshot): answer the review round on sharing, parity, and evidence The eager-closure refusal the review asked to be pointed at or filed is not a table of budgets: the gate ratchets every entry's closure against the committed merge-base tree, so the numbers to read are 9 for `facades/capture.ts` and 2 for `snapshot-quality-verdict.ts`. Re-installing one shared kernel predicate for both readers on this head fails six entries and names both of those, which is why the vocabulary stays a map per reader; recorded on #2872 as the accepted deviation. The `android-helper` asymmetry is not real: `SNAPSHOT_QUALITY_BACKEND_CAPABILITIES` merges the Android declaration, so both readers accept that strategy and the parity table now carries the row instead of omitting it. The client-side drop of an unknown state or backend gets its CHANGELOG entry, the justification comments shrink to one constraint sentence each, the kernel test keeps only its `@ts-expect-error`, and the word left dangling by the last rename goes.
1 parent 3e5af3a commit d7a1f2c

13 files changed

Lines changed: 351 additions & 69 deletions

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -535,7 +535,7 @@ extension RunnerTests {
535535
return sparseTruncatedSnapshotPayload(
536536
message: recoveredSnapshotMessage(failure),
537537
snapshotQuality: SnapshotQuality(
538-
state: "sparse",
538+
state: .sparse,
539539
backend: SnapshotBackendKind.recursiveTree.rawValue,
540540
reason: failure.message,
541541
reasonCode: "ax-rejected",

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotCapturePlan.swift‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,21 @@ import AgentDeviceSnapshotPresentation
88
// stamp the outcome with a structured quality verdict so the daemon renders state instead of
99
// re-deriving it from node shapes. Recovery ordering is data here, never a per-call-site branch.
1010

11+
/// The closed set of verdict states the host accepts. The wire strings are the shared table at
12+
/// `contracts/fixtures/ios-snapshot-quality-states.json`, which `allCases` is pinned to; `reasonCode`
13+
/// stays open because an unknown one costs only its wording, never the verdict.
14+
enum SnapshotQualityState: String, Codable, CaseIterable {
15+
/// First backend produced a usable tree.
16+
case healthy
17+
/// A later backend did.
18+
case recovered
19+
/// No backend produced a usable tree; the best attempt is returned as-is.
20+
case sparse
21+
}
22+
1123
/// Structured quality verdict shipped with every iOS snapshot payload.
1224
struct SnapshotQuality: Codable {
13-
/// healthy: first backend produced a usable tree. recovered: a later backend did.
14-
/// sparse: no backend produced a usable tree; the best attempt is returned as-is.
15-
let state: String
25+
let state: SnapshotQualityState
1626
/// Backend that produced the returned payload: tree | queries | private-ax.
1727
let backend: String
1828
/// Why recovery ran (first failure), why the payload is degraded, or why an internal backend
@@ -391,7 +401,7 @@ extension RunnerTests {
391401
return stampedSnapshotPayload(
392402
capture,
393403
backend: kind,
394-
state: recovered ? "recovered" : "healthy",
404+
state: recovered ? .recovered : .healthy,
395405
reason: recovered || firstFailure?.code == "requested-backend" ? firstFailure : nil
396406
)
397407
}
@@ -416,11 +426,11 @@ extension RunnerTests {
416426
}
417427

418428
let fallbackPayload =
419-
best.map { stampedSnapshotPayload($0.capture, backend: $0.kind, state: "sparse", reason: firstFailure) }
429+
best.map { stampedSnapshotPayload($0.capture, backend: $0.kind, state: .sparse, reason: firstFailure) }
420430
?? stampedSnapshotPayload(
421431
SnapshotBackendCapture(payload: sparseTruncatedSnapshotPayload(), effectiveDepth: nil),
422432
backend: effectivePlan.last ?? plan.last ?? .recursiveTree,
423-
state: "sparse",
433+
state: .sparse,
424434
reason: firstFailure
425435
)
426436
return fallbackPayload
@@ -679,7 +689,7 @@ extension RunnerTests {
679689
func stampedSnapshotPayload(
680690
_ capture: SnapshotBackendCapture,
681691
backend: SnapshotBackendKind,
682-
state: String,
692+
state: SnapshotQualityState,
683693
reason: (reason: String, code: String)?
684694
) -> DataPayload {
685695
let health: RunnerAccessibilityHealth = reason?.code == "ax-rejected" ? .unavailable : .healthy
@@ -704,7 +714,7 @@ extension RunnerTests {
704714
// "recovered") stays untruncated, so strict absence reads can trust it. Only a real cap
705715
// (payload truncation, a depth-limited private AX capture) or a sparse terminal payload
706716
// is truncated.
707-
truncated: payload.truncated == true || state == "sparse" || capture.effectiveDepth != nil,
717+
truncated: payload.truncated == true || state == .sparse || capture.effectiveDepth != nil,
708718
qualityPayload: capture.qualityPayload.flatMap { quality in
709719
guard let nodes = quality.nodes else { return nil }
710720
return SnapshotQualityPayload(nodes: nodes, truncated: quality.truncated == true)

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+SnapshotCapturePlanOccupancyTests.swift‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ extension RunnerTests {
148148
XCTAssertNil(box.error)
149149
let quality = try XCTUnwrap(box.payload?.snapshotQuality)
150150
XCTAssertEqual(quality.backend, SnapshotBackendKind.privateAX.rawValue)
151-
XCTAssertEqual(quality.state, "recovered")
151+
XCTAssertEqual(quality.state, .recovered)
152152
XCTAssertTrue(
153153
quality.reason?.contains("tree capture exceeded") == true,
154154
"the tree XPC, not the viewport read, must be the abandoned block: \(quality.reason ?? "nil")"
@@ -344,7 +344,7 @@ extension RunnerTests {
344344
SnapshotBackendKind.privateAX.rawValue,
345345
"a sweep that ended on its slice deadline is a tier timeout, not an accepted capture"
346346
)
347-
XCTAssertEqual(quality?.state, "recovered")
347+
XCTAssertEqual(quality?.state, .recovered)
348348
XCTAssertGreaterThan(box.payload?.nodes?.count ?? 0, 1, "private AX answers with a real tree")
349349
XCTAssertFalse(box.abandonedAtReturn, "the sweep must answer inside its own main-thread hop")
350350
XCTAssertTrue(

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+SnapshotCapturePlanTests.swift‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ extension RunnerTests {
162162
let payload = stampedSnapshotPayload(
163163
capture,
164164
backend: .recursiveTree,
165-
state: "healthy",
165+
state: .healthy,
166166
reason: nil
167167
)
168168

@@ -187,7 +187,7 @@ extension RunnerTests {
187187
customActions: coverage
188188
),
189189
backend: .recursiveTree,
190-
state: "healthy",
190+
state: .healthy,
191191
reason: nil
192192
)
193193
XCTAssertNil(silent.message)
@@ -200,7 +200,7 @@ extension RunnerTests {
200200
effectiveDepth: 4
201201
),
202202
backend: .privateAX,
203-
state: "recovered",
203+
state: .recovered,
204204
reason: (reason: "tree capture timed out", code: "budget")
205205
)
206206
XCTAssertEqual(underlying.message, "underlying")
@@ -224,24 +224,24 @@ extension RunnerTests {
224224
// The CI signature behind `is absent ... capture was truncated`: a complete private AX
225225
// tree selected while the XCTest channel is penalized is whole, and must say so.
226226
let recovered = stampedSnapshotPayload(
227-
complete, backend: .privateAX, state: "recovered", reason: deferred)
228-
XCTAssertEqual(recovered.snapshotQuality?.state, "recovered")
227+
complete, backend: .privateAX, state: .recovered, reason: deferred)
228+
XCTAssertEqual(recovered.snapshotQuality?.state, .recovered)
229229
XCTAssertEqual(recovered.truncated, false)
230230

231231
let depthLimited = stampedSnapshotPayload(
232232
SnapshotBackendCapture(payload: complete.payload, effectiveDepth: 56),
233-
backend: .privateAX, state: "recovered", reason: deferred)
233+
backend: .privateAX, state: .recovered, reason: deferred)
234234
XCTAssertEqual(depthLimited.truncated, true)
235235

236236
let cappedPayload = stampedSnapshotPayload(
237237
SnapshotBackendCapture(
238238
payload: DataPayload(nodes: complete.payload.nodes ?? [], truncated: true),
239239
effectiveDepth: nil),
240-
backend: .recursiveTree, state: "healthy", reason: nil)
240+
backend: .recursiveTree, state: .healthy, reason: nil)
241241
XCTAssertEqual(cappedPayload.truncated, true)
242242

243243
let sparse = stampedSnapshotPayload(
244-
complete, backend: .querySweep, state: "sparse",
244+
complete, backend: .querySweep, state: .sparse,
245245
reason: ("snapshot returned no semantic controls or content", "sparse-tree"))
246246
XCTAssertEqual(sparse.truncated, true)
247247
}
@@ -260,7 +260,7 @@ extension RunnerTests {
260260
let payload = stampedSnapshotPayload(
261261
capture,
262262
backend: .recursiveTree,
263-
state: "healthy",
263+
state: .healthy,
264264
reason: nil
265265
)
266266

@@ -291,7 +291,7 @@ extension RunnerTests {
291291
let payload = stampedSnapshotPayload(
292292
capture,
293293
backend: .recursiveTree,
294-
state: "healthy",
294+
state: .healthy,
295295
reason: nil
296296
)
297297

@@ -507,7 +507,7 @@ extension RunnerTests {
507507

508508
let quality = try XCTUnwrap(capped.snapshotQuality)
509509
XCTAssertEqual(quality.backend, SnapshotBackendKind.privateAX.rawValue)
510-
XCTAssertNotEqual(quality.state, "sparse")
510+
XCTAssertNotEqual(quality.state, .sparse)
511511
let nodes = try XCTUnwrap(capped.nodes)
512512
XCTAssertGreaterThan(nodes.count, 1)
513513
XCTAssertEqual(nodes.map(\.depth).max(), 1)
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
import XCTest
2+
3+
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
4+
extension RunnerTests {
5+
private struct StampedWireVerdict: Decodable {
6+
struct Quality: Decodable {
7+
let state: String
8+
}
9+
let snapshotQuality: Quality
10+
}
11+
12+
private func loadSnapshotQualityStatesFixture() throws -> [String] {
13+
let fixtureURL = URL(fileURLWithPath: #filePath)
14+
.deletingLastPathComponent() // UnitTests
15+
.deletingLastPathComponent() // AgentDeviceRunnerUITests
16+
.deletingLastPathComponent() // AgentDeviceRunner
17+
.deletingLastPathComponent() // runner
18+
.deletingLastPathComponent() // apple
19+
.deletingLastPathComponent() // repo root
20+
.appendingPathComponent("contracts")
21+
.appendingPathComponent("fixtures")
22+
.appendingPathComponent("ios-snapshot-quality-states.json")
23+
return try JSONDecoder().decode([String].self, from: Data(contentsOf: fixtureURL))
24+
}
25+
26+
/// The one claim of this file: the runner's closed enum and the shared TypeScript table name the
27+
/// same states. The kernel's `SNAPSHOT_QUALITY_STATES` is pinned to it too, so the two runtimes
28+
/// cannot drift into a verdict the host drops along with its disclosure. Compared as a set: the
29+
/// names are the contract, and a reordering of `allCases` cannot produce a wrong verdict.
30+
func testSnapshotQualityStatesMatchSharedWireFixture() throws {
31+
XCTAssertEqual(
32+
Set(try loadSnapshotQualityStatesFixture()),
33+
Set(SnapshotQualityState.allCases.map(\.rawValue)),
34+
"update the fixture and the kernel tuple together with the enum"
35+
)
36+
}
37+
38+
/// What the daemon receives for each state, taken from the production stamping path rather than a
39+
/// hand-built verdict: the wire string is the case's own raw value, so a change of
40+
/// representation — an `Int` backing, a nested object — goes red here on the actual payload, and
41+
/// a renamed raw value goes red in the fixture test above.
42+
func testStampedVerdictEncodesTheCaseRawValue() throws {
43+
let capture = SnapshotBackendCapture(
44+
payload: DataPayload(nodes: [], truncated: false),
45+
effectiveDepth: nil
46+
)
47+
for state in SnapshotQualityState.allCases {
48+
let payload = stampedSnapshotPayload(
49+
capture,
50+
backend: .recursiveTree,
51+
state: state,
52+
reason: nil
53+
)
54+
let wire = try JSONDecoder().decode(
55+
StampedWireVerdict.self,
56+
from: JSONEncoder().encode(payload)
57+
)
58+
XCTAssertEqual(wire.snapshotQuality.state, state.rawValue)
59+
}
60+
}
61+
62+
/// Closed in both directions: a wire string nobody declared never becomes a verdict.
63+
func testVerdictStateRejectsAnUndeclaredWireString() throws {
64+
let json = Data(#"{"state":"degraded","backend":"tree"}"#.utf8)
65+
XCTAssertThrowsError(try JSONDecoder().decode(SnapshotQuality.self, from: json))
66+
}
67+
}
68+
#endif

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+SnapshotTests.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ extension RunnerTests {
2323
XCTAssertEqual(payload.runnerFatalReason, Self.axSnapshotUnavailableReason)
2424
// The planned terminal result carries the structured verdict like every other planned
2525
// snapshot — downstream sparse handling keys off it, not off node shapes.
26-
XCTAssertEqual(payload.snapshotQuality?.state, "sparse")
26+
XCTAssertEqual(payload.snapshotQuality?.state, .sparse)
2727
XCTAssertEqual(payload.snapshotQuality?.reasonCode, "ax-rejected")
2828
XCTAssertEqual(payload.snapshotQuality?.reason, Self.axSnapshotFailureMessage)
2929
XCTAssertNil(currentApp)
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
["healthy", "recovered", "sparse"]

‎packages/capture-kit/src/snapshot-quality-verdict.test.ts‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import { test } from 'vitest';
22
import assert from 'node:assert/strict';
33

4+
import { readSerializedSnapshotCaptureAnnotations } from '@agent-device/contracts/capture';
5+
import { SNAPSHOT_QUALITY_STATES } from '@agent-device/kernel/snapshot';
46
import {
57
isSparseSnapshotQualityVerdict,
68
preferredSnapshotBackendForVerdict,
@@ -47,6 +49,22 @@ test('readSnapshotQualityVerdict rejects unknown state or backend as verdict-abs
4749
assert.equal(readSnapshotQualityVerdict({ state: 'sparse', backend: 'mystery' }), undefined);
4850
assert.equal(readSnapshotQualityVerdict({ backend: 'tree' }), undefined);
4951
assert.equal(readSnapshotQualityVerdict(null), undefined);
52+
// An inherited key is not a declared state: membership stays on the map's own keys.
53+
assert.equal(readSnapshotQualityVerdict({ state: 'constructor', backend: 'tree' }), undefined);
54+
});
55+
56+
test('readSnapshotQualityVerdict reads every declared wire state', () => {
57+
for (const state of SNAPSHOT_QUALITY_STATES) {
58+
assert.deepEqual(readSnapshotQualityVerdict({ state, backend: 'tree' }), {
59+
state,
60+
backend: 'tree',
61+
reason: undefined,
62+
reasonCode: undefined,
63+
customActions: undefined,
64+
effectiveDepth: undefined,
65+
collapsedLeafIndexes: undefined,
66+
});
67+
}
5068
});
5169

5270
test('readSnapshotQualityVerdict keeps the verdict but drops an unknown reasonCode', () => {
@@ -98,3 +116,43 @@ test('preferredSnapshotBackendForVerdict pins only private-ax captures', () => {
98116
);
99117
assert.equal(preferredSnapshotBackendForVerdict(undefined), undefined);
100118
});
119+
120+
/**
121+
* Two readings of one verdict exist on purpose: this module normalizes an untrusted runner payload,
122+
* while contracts re-publishes what this repo published and normalizes nothing (the eager-closure
123+
* gate forbids either reaching a shared module, and the duplication gate refuses a second
124+
* normalization). They must still agree on which payloads are a verdict at all: a name one version
125+
* cannot speak is verdict-absent on both sides of the daemon boundary.
126+
*/
127+
const VERDICT_PAYLOADS: unknown[] = [
128+
{ state: 'sparse', backend: 'private-ax' },
129+
{ state: 'healthy', backend: 'tree', reason: 'ok', reasonCode: 'requested-backend' },
130+
{ state: 'recovered', backend: 'queries', reason: 42, effectiveDepth: '56' },
131+
{ state: 'sparse', backend: 'tree', collapsedLeafIndexes: [3, 'four'] },
132+
{ state: 'sparse', backend: 'tree', customActions: { read: 12 } },
133+
{ state: 'sparse', backend: 'tree', customActions: { read: 12, candidates: 19 } },
134+
{ state: 'sparse', backend: 'tree', timing: { acquisitionMs: 12.5 } },
135+
{ state: 'sparse', backend: 'tree', timing: { acquisitionMs: 12.5, presentationMs: 34.75 } },
136+
{ state: 'sparse', backend: 'tree', reasonCode: 'future-code' },
137+
{ state: 'recovered', backend: 'android-helper', reasonCode: 'requested-backend' },
138+
{ state: 'degraded', backend: 'tree' },
139+
{ state: 'sparse', backend: 'uiautomator' },
140+
{ state: 'constructor', backend: 'constructor' },
141+
{ backend: 'tree' },
142+
{ state: 'sparse' },
143+
null,
144+
'verdict',
145+
];
146+
147+
test('the contracts re-read calls a verdict a verdict on every payload', () => {
148+
for (const payload of VERDICT_PAYLOADS) {
149+
const reRead = readSerializedSnapshotCaptureAnnotations({
150+
snapshotQuality: payload,
151+
}).snapshotQuality;
152+
assert.equal(
153+
reRead === undefined,
154+
readSnapshotQualityVerdict(payload) === undefined,
155+
JSON.stringify(payload),
156+
);
157+
}
158+
});

0 commit comments

Comments
 (0)