Skip to content

Commit 7a5b4eb

Browse files
authored
fix(selectors): keep downgraded target annotations under the payload ceiling (callstack#4354)
* fix(selectors): keep downgraded target annotations under the payload ceiling Field caps count raw UTF-8 bytes while the 4096-byte ceiling counts escaped JSON, so escape-heavy fields could leave an unverifiable annotation that its own parser rejects. After the downgrade the writer now drops rect, scrollRegion, the remaining ancestry, then label, measuring after each drop. Replay refuses recorded unverifiable evidence before reading these fields, and role plus id always fit. Fixes callstack#4333 * fix(selectors): check the payload ceiling after the last downgrade drop The writer now measures again after dropping label and throws if role and id still exceed the ceiling, instead of returning a payload its parser rejects. The role-and-id invariant test derives its bound from the field cap and the largest JSON escape expansion rather than a hand-built payload.
1 parent 0db888a commit 7a5b4eb

4 files changed

Lines changed: 159 additions & 9 deletions

File tree

‎docs/adr/0012-interactive-replay.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ An inserted or removed wrapper ancestor changes identity.
6666
Replay classifies every annotated step before dispatch through six exact verification paths.
6767
Only a verified result sends the action.
6868
Anything that the evidence cannot isolate is an `identity-unverifiable` divergence, never a silent pick.
69-
The recorder must never emit a payload that its parser rejects. The writer downgrades oversized evidence to `verification: "unverifiable"` and drops its rect. Escape-heavy values can still exceed the serialized limit; the implementation does not recheck after dropping the rect.
69+
The recorder must never emit a payload that its parser rejects. The writer downgrades oversized evidence to `verification: "unverifiable"`, then drops optional fields and measures again after each drop until the serialized payload fits.
7070
The format is additive. Old readers ignore the comment, and new readers run unannotated actions without the check.
7171
An annotation binds only to the next physical action line.
7272

‎packages/ad-script/docs/target-annotation.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,12 @@ If per-field truncation still leaves more than 4 KiB, the writer removes `ancest
6666
This reduction is deterministic and uses the existing ancestry truncation direction.
6767
If only `ancestry[0]` remains and the payload still exceeds the limit, the writer sets `verification: "unverifiable"`.
6868
The limit measures serialized UTF-8 bytes, while field limits apply before JSON escaping.
69-
The writer drops `rect` if the downgraded payload still exceeds the limit, but does not recheck afterward.
70-
Escape-heavy values can therefore still produce an over-limit annotation that the parser rejects.
69+
Escape-heavy values can still exceed the limit after this downgrade.
70+
The writer then drops `rect`, `scrollRegion`, the remaining `ancestry`, and `label`, in that order.
71+
It measures the payload again after each drop and stops when the payload fits.
72+
Replay refuses recorded `"unverifiable"` evidence before it reads these fields, so the drops do not weaken replay.
73+
A capped field escapes to at most six times its limit, so `role` and `id` alone always fit and the writer never drops them.
74+
If a field limit change breaks that bound, the writer throws instead of writing a payload that exceeds the limit.
7175
When the reduced tuple fits, the record-time self-check uses that tuple.
7276
A `verified` claim therefore applies to exactly the written evidence.
7377

‎packages/selectors/src/target-evidence.test.ts‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,137 @@ test('computeTargetEvidence: max-size labels across K=8 ancestors reduce ancestr
221221
assert.deepEqual(reparsed, evidence);
222222
});
223223

224+
// #4333: field caps count raw UTF-8 bytes, but the payload ceiling counts the serialized JSON. A
225+
// control character is one raw byte and six escaped ones (`\u0001`), so fields at their cap can
226+
// still overflow after the writer has dropped ancestry to its floor and dropped `rect`.
227+
const ESCAPE_HEAVY_FIELD = '\u0001'.repeat(TARGET_ANNOTATION_MAX_FIELD_BYTES);
228+
/** JSON bytes per raw byte for a one-byte control character, the largest expansion JSON escaping makes. */
229+
const MAX_JSON_ESCAPE_EXPANSION = JSON.stringify('\u0001').length - '""'.length;
230+
231+
test('computeTargetEvidence: escape-heavy fields degrade until the unverifiable payload fits and reparses', () => {
232+
const nodes = toSnapshotNodes([
233+
{ index: 0, type: ESCAPE_HEAVY_FIELD, label: ESCAPE_HEAVY_FIELD, depth: 0 },
234+
{
235+
index: 1,
236+
type: ESCAPE_HEAVY_FIELD,
237+
label: ESCAPE_HEAVY_FIELD,
238+
identifier: ESCAPE_HEAVY_FIELD,
239+
rect: { x: 0, y: 0, width: 10, height: 10 },
240+
depth: 1,
241+
parentIndex: 0,
242+
},
243+
]);
244+
const winner = nodes.at(-1)!;
245+
246+
const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes });
247+
248+
assert.ok(evidence);
249+
assert.equal(evidence.verification, 'unverifiable');
250+
const json = serializeTargetAnnotationV1(evidence);
251+
assert.ok(
252+
utf8ByteLength(json) <= TARGET_ANNOTATION_MAX_PAYLOAD_BYTES,
253+
`payload is ${utf8ByteLength(json)} bytes`,
254+
);
255+
assert.deepEqual(parseTargetAnnotationV1Payload(json), evidence);
256+
});
257+
258+
// #4333: the downgrade drops optional fields in a fixed order and stops at the first payload that
259+
// fits. Sweeping the winner's escape-heavy label across the ceiling passes every stopping point, so
260+
// no stopping point needs its own tuned length. The scroll container's label is short enough that
261+
// the winner fits whole without a label, and long enough that scrollRegion and ancestry each stop a
262+
// sweep.
263+
const OVERSIZED_DROP_ORDER = ['rect', 'scrollRegion', 'ancestry', 'label'] as const;
264+
const SCROLL_CONTAINER = { role: 'scrollview', label: '\u0001'.repeat(50) };
265+
266+
test('computeTargetEvidence: an oversized payload drops rect, scrollRegion, ancestry, then label, and stops once it fits', () => {
267+
const rect = { x: 0, y: 0, width: 10, height: 10 };
268+
const build = (labelLength: number) =>
269+
toSnapshotNodes([
270+
{ index: 0, type: 'Window', depth: 0 },
271+
{ index: 1, type: 'ScrollView', label: SCROLL_CONTAINER.label, depth: 1, parentIndex: 0 },
272+
{
273+
index: 2,
274+
type: ESCAPE_HEAVY_FIELD,
275+
identifier: ESCAPE_HEAVY_FIELD,
276+
label: '\u0001'.repeat(labelLength),
277+
rect,
278+
depth: 2,
279+
parentIndex: 1,
280+
},
281+
]);
282+
const unlabelled = build(0);
283+
const whole = computeTargetEvidence({ node: unlabelled.at(-1)!, preActionNodes: unlabelled });
284+
assert.equal(whole?.verification, 'verified');
285+
assert.deepEqual(whole.scrollRegion, SCROLL_CONTAINER);
286+
assert.deepEqual(whole.ancestry[0], SCROLL_CONTAINER);
287+
const isPresent = (evidence: typeof whole, field: (typeof OVERSIZED_DROP_ORDER)[number]) =>
288+
field === 'ancestry' ? evidence.ancestry.length > 0 : evidence[field] !== undefined;
289+
290+
const stops = new Set<number>();
291+
for (let labelLength = 1; labelLength <= TARGET_ANNOTATION_MAX_FIELD_BYTES; labelLength += 1) {
292+
const nodes = build(labelLength);
293+
const evidence = computeTargetEvidence({ node: nodes.at(-1)!, preActionNodes: nodes });
294+
assert.ok(evidence);
295+
const firstKept = OVERSIZED_DROP_ORDER.findIndex((field) => isPresent(evidence, field));
296+
const droppedCount = firstKept === -1 ? OVERSIZED_DROP_ORDER.length : firstKept;
297+
for (const field of OVERSIZED_DROP_ORDER.slice(droppedCount)) {
298+
assert.ok(
299+
isPresent(evidence, field),
300+
`${field} dropped out of order at length ${labelLength}`,
301+
);
302+
}
303+
if (droppedCount > 0) {
304+
// The values each field held before the downgrade; ancestry was already at its floor.
305+
const undropped = {
306+
rect,
307+
scrollRegion: SCROLL_CONTAINER,
308+
ancestry: [SCROLL_CONTAINER],
309+
label: '\u0001'.repeat(labelLength),
310+
};
311+
const lastDropped = OVERSIZED_DROP_ORDER[droppedCount - 1]!;
312+
const restored = Object.assign(structuredClone(evidence), {
313+
[lastDropped]: undropped[lastDropped],
314+
});
315+
assert.ok(
316+
utf8ByteLength(serializeTargetAnnotationV1(restored)) > TARGET_ANNOTATION_MAX_PAYLOAD_BYTES,
317+
`${lastDropped} dropped at length ${labelLength} although the payload already fit`,
318+
);
319+
}
320+
const json = serializeTargetAnnotationV1(evidence);
321+
assert.ok(utf8ByteLength(json) <= TARGET_ANNOTATION_MAX_PAYLOAD_BYTES);
322+
assert.deepEqual(parseTargetAnnotationV1Payload(json), evidence);
323+
stops.add(droppedCount);
324+
}
325+
assert.deepEqual(
326+
[...stops].sort((a, b) => a - b),
327+
OVERSIZED_DROP_ORDER.map((_, index) => index).concat(OVERSIZED_DROP_ORDER.length),
328+
'the sweep must stop once with nothing dropped and once after each field',
329+
);
330+
});
331+
332+
// The downgrade never drops `role` or `id`, so it relies on the two fitting even fully escaped. A
333+
// larger field cap breaks this, and the writer would again emit a payload its parser rejects.
334+
test('computeTargetEvidence: a downgrade with every optional field dropped fits at the worst-case escape ratio', () => {
335+
const skeleton = serializeTargetAnnotationV1({
336+
id: '',
337+
role: '',
338+
ancestry: [],
339+
sibling: Number.MAX_SAFE_INTEGER,
340+
viewportOrder: Number.MAX_SAFE_INTEGER,
341+
verification: 'unverifiable',
342+
});
343+
const retainedFields = ['id', 'role'].length;
344+
345+
const worstCaseBytes =
346+
utf8ByteLength(skeleton) +
347+
retainedFields * MAX_JSON_ESCAPE_EXPANSION * TARGET_ANNOTATION_MAX_FIELD_BYTES;
348+
349+
assert.ok(
350+
worstCaseBytes <= TARGET_ANNOTATION_MAX_PAYLOAD_BYTES,
351+
`role and id at the field cap escape to a ${worstCaseBytes}-byte payload`,
352+
);
353+
});
354+
224355
test('computeTargetEvidence: a root node with no ancestors has empty ancestry and is still verified', () => {
225356
const nodes = toSnapshotNodes([
226357
{

‎packages/selectors/src/target-evidence.ts‎

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -151,15 +151,30 @@ function fitsPayloadCeiling(candidate: TargetAnnotationV1): boolean {
151151
);
152152
}
153153

154-
/** Decision 3's terminal fail-closed downgrade; rect is diagnostic-only and dropped before emitting an over-cap payload. */
154+
/**
155+
* Field caps count raw bytes and the ceiling counts escaped JSON, so dropping ancestry to its
156+
* floor can still leave an over-cap payload. Replay refuses recorded-unverifiable evidence before
157+
* it reads any other field, so each optional field drops in turn until the payload fits. A capped
158+
* field escapes to at most six times its cap, so `role` and `id` alone always fit and never drop.
159+
*/
160+
const OVERSIZED_CANDIDATE_DEGRADATIONS: readonly ((candidate: TargetAnnotationV1) => void)[] = [
161+
(candidate) => delete candidate.rect,
162+
(candidate) => delete candidate.scrollRegion,
163+
(candidate) => (candidate.ancestry = []),
164+
(candidate) => delete candidate.label,
165+
];
166+
167+
/** Decision 3's terminal fail-closed downgrade: never emit a payload the parser rejects. */
155168
function finalizeOversizedCandidate(candidate: TargetAnnotationV1): TargetAnnotationV1 {
156169
candidate.verification = 'unverifiable';
157-
if (
158-
utf8ByteLength(serializeTargetAnnotationV1(candidate)) > TARGET_ANNOTATION_MAX_PAYLOAD_BYTES
159-
) {
160-
delete candidate.rect;
170+
for (const degrade of OVERSIZED_CANDIDATE_DEGRADATIONS) {
171+
if (fitsPayloadCeiling(candidate)) return candidate;
172+
degrade(candidate);
161173
}
162-
return candidate;
174+
if (fitsPayloadCeiling(candidate)) return candidate;
175+
throw new Error(
176+
'target annotation exceeds the payload ceiling with only role and id left; the field cap no longer fits the ceiling',
177+
);
163178
}
164179

165180
// ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)