Skip to content

Commit 2149096

Browse files
committed
feat: --settle returns the settled diff in the interaction response (#1101)
press/click/fill/longpress --settle executes the action, waits for the UI to go quiet (wait stable's loop, shared via stable-capture.ts), and returns the settled diff vs the pre-action tree in the same response — one round trip instead of the interact -> observe pair. - payload: changed lines only (bounded), summary counts, added-line refs, refsGeneration; best-effort (settled:false + hint on never-quiet content, never an action failure); --verify shares the settle captures - ref issuance: the settled tree becomes the session snapshot; a diff-carrying settle response clears snapshotRefsStale and the MCP layer merge-only re-pins added-line refs at the settle generation - grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced descriptor budget with new envelope:'widen' semantics mirroring wait) - ADR 0011: new settleObservation guarantee classified on every path with contract scenarios per enforced/delegated cell
1 parent ea69dc3 commit 2149096

48 files changed

Lines changed: 2143 additions & 236 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CONTEXT.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
- Coverage manifest: `CONTRACT_COVERAGE` export beside each interaction contract test file claiming which matrix cells it proves; the coverage gate requires every enforced/delegated cell to be claimed and rejects overclaims of waived cells.
5050
- Delegation-on-error: a fast path falling back to the runtime path on semantic failure shapes. It closes failure-side guarantee cells only — never success-path parity.
5151
- Ref generation pin: optional `~s<n>` suffix on an @ref carrying the snapshot generation it was minted from. Accepted as input everywhere, emitted by no tree output (snapshot token budget), auto-appended by the MCP layer, stripped and ignored by replay.
52+
- Settled observation: opt-in (`--settle`) post-action payload on press/click/fill/longpress — the quiet-window stable loop re-captures until the UI settles, and the response carries the diff vs the pre-action tree (changed lines only, added lines with fresh refs, `refsGeneration` when the settled tree was stored). Best-effort: never fails the action; `settled: false` plus a hint on never-quiet content.
5253
- Snapshot capture plan: per-strategy ordered chain of iOS snapshot capture backends (recursive tree, query sweep, private AX) run by one plan runner under a shared wall-clock budget; recovery ordering is declared data, never a per-call-site branch.
5354
- Snapshot quality verdict: structured outcome (state, backend, reason code, effective depth, collapsed leaves) computed once by the plan runner and shipped with every planned snapshot payload; the daemon and CLI render it instead of re-deriving degradation from node shapes.
5455
- AX-unavailable target invalidation: iOS/macOS runner behavior where a root accessibility snapshot failure such as `kAXErrorIllegalArgument` marks the cached `XCUIApplication` target handle suspect. The runner fails closed for degraded interactive snapshots, clears the cached target, and lets the next command reacquire the app through normal activation.

‎scripts/integration-progress-model.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,8 @@ function summarizeProviderScenarioFlagCoverage(files) {
185185
['findFirst', 'find first disambiguation'],
186186
['findLast', 'find last disambiguation'],
187187
['verify', 'post-action evidence capture on press/click/fill'],
188+
['settle', 'post-action settled-diff observation on press/click/fill/longpress'],
189+
['settleQuietMs', 'settle quiet-window tuning'],
188190
];
189191
const sources = files.map((file) => fs.readFileSync(file, 'utf8')).join('\n');
190192
return flagTargets.map(([key, reason, aliases = []]) => {

‎src/cli/parser/cli-flags.ts‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,8 @@ export type CliFlags = CloudProviderProfileFields &
106106
pixels?: number;
107107
doubleTap?: boolean;
108108
verify?: boolean;
109+
settle?: boolean;
110+
settleQuietMs?: number;
109111
clickButton?: ClickButton;
110112
backMode?: BackMode;
111113
pauseMs?: number;
@@ -211,6 +213,10 @@ export const REPEATED_TOUCH_FLAGS = flagKeys(
211213
'jitterPx',
212214
'doubleTap',
213215
);
216+
// press/click/fill/longpress --settle (#1101): opt-in settled-diff observation.
217+
// --timeout doubles as the settle deadline (flag-sourced budget on the
218+
// interaction descriptors, mirroring wait's positional budget).
219+
export const SETTLE_FLAGS = flagKeys('settle', 'settleQuietMs', 'timeoutMs');
214220
export const REPLAY_FLAGS = flagKeys('replayUpdate', 'replayEnv');
215221

216222
const FLAG_DEFINITIONS: readonly FlagDefinition[] = [
@@ -848,6 +854,22 @@ const FLAG_DEFINITIONS: readonly FlagDefinition[] = [
848854
usageDescription:
849855
'Capture cheap post-action evidence (AX digest, node counts, changedFromBefore) instead of a follow-up snapshot',
850856
},
857+
{
858+
key: 'settle',
859+
names: ['--settle'],
860+
type: 'boolean',
861+
usageLabel: '--settle',
862+
usageDescription:
863+
'After the action, wait for the UI to go quiet and return the settled diff vs the pre-action tree in the same response (best-effort; never fails the action)',
864+
},
865+
{
866+
key: 'settleQuietMs',
867+
names: ['--settle-quiet'],
868+
type: 'int',
869+
min: 0,
870+
usageLabel: '--settle-quiet <ms>',
871+
usageDescription: 'Settle: quiet window the UI must hold to count as settled (default 500ms)',
872+
},
851873
{
852874
key: 'clickButton',
853875
names: ['--button'],
@@ -1051,7 +1073,7 @@ const FLAG_DEFINITIONS: readonly FlagDefinition[] = [
10511073
min: 1,
10521074
usageLabel: '--timeout <ms>',
10531075
usageDescription:
1054-
'Prepare/Replay/Snapshot/Test: maximum wall-clock time for the command or attempt',
1076+
'Prepare/Replay/Snapshot/Test: maximum wall-clock time for the command or attempt. With --settle: the settle-wait deadline (default 10s)',
10551077
},
10561078
{
10571079
key: 'retries',

‎src/cli/parser/cli-help.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ const AGENT_QUICKSTART_LINES = [
6161
'Planning output contract: when asked to plan commands, output command lines only: no prose, numbering, Markdown fences, pipes, or shell helpers.',
6262
'Default loop: devices/apps -> open -> snapshot -i -> press/fill/get/is/wait/find -> verify with diff snapshot -> close.',
6363
'Verify a mutation with diff snapshot (or diff snapshot -i), not a full snapshot: it prints only the added/removed/changed lines since the last snapshot in this session, so confirming an action costs a few lines instead of the whole tree.',
64+
'Collapse act+observe into one call with --settle on press/click/fill/longpress: the response waits for the UI to go quiet and carries the settled diff with fresh refs (settled: false plus a hint on never-quiet content; the action itself never fails). Tune with --settle-quiet <ms> and --timeout <ms>.',
6465
'Use selectors or refs as positional targets: id="submit", label="Allow", or @e12 from snapshot -i.',
6566
'Pin a ref to the snapshot that minted it with ~s<n> (n = refsGeneration in the snapshot response): press @e12~s4. Pinned refs get exact staleness warnings instead of the coarse tree-changed one; plain refs stay valid input.',
6667
'Plain snapshot reads state; snapshot -i refreshes current interactive refs only.',

‎src/client/client-normalizers.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,8 @@ export function buildFlags(options: InternalRequestOptions): CommandFlags {
343343
pixels: options.pixels,
344344
doubleTap: options.doubleTap,
345345
verify: options.verify,
346+
settle: options.settle,
347+
settleQuietMs: options.settleQuietMs,
346348
clickButton: options.clickButton,
347349
pauseMs: options.pauseMs,
348350
pattern: options.pattern,

‎src/client/client-types.ts‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -629,10 +629,24 @@ export type CaptureDiffOptions = DeviceCommandBaseOptions &
629629
out?: string;
630630
};
631631

632+
/**
633+
* Opt-in (#1101): after the action, wait for the UI to go quiet and return the
634+
* settled diff vs the pre-action tree (`settle` on the result) in the same
635+
* response. Best-effort — never fails the action. `settleQuietMs` tunes the
636+
* quiet window (default 500ms); `timeoutMs` bounds the settle wait (default
637+
* 10s) and is rejected without `settle`.
638+
*/
639+
type SettleCommandOptions = {
640+
settle?: boolean;
641+
settleQuietMs?: number;
642+
timeoutMs?: number;
643+
};
644+
632645
export type ClickOptions = DeviceCommandBaseOptions &
633646
SelectorSnapshotCommandOptions &
634647
InteractionTarget &
635-
RepeatedPressOptions & {
648+
RepeatedPressOptions &
649+
SettleCommandOptions & {
636650
button?: ClickButton;
637651
/**
638652
* Opt-in (#1047): return cheap post-action evidence (AX digest, node counts,
@@ -645,13 +659,15 @@ export type ClickOptions = DeviceCommandBaseOptions &
645659
export type PressOptions = DeviceCommandBaseOptions &
646660
SelectorSnapshotCommandOptions &
647661
InteractionTarget &
648-
RepeatedPressOptions & {
662+
RepeatedPressOptions &
663+
SettleCommandOptions & {
649664
verify?: boolean;
650665
};
651666

652667
export type LongPressOptions = DeviceCommandBaseOptions &
653668
SelectorSnapshotCommandOptions &
654-
InteractionTarget & {
669+
InteractionTarget &
670+
SettleCommandOptions & {
655671
durationMs?: number;
656672
};
657673

@@ -697,7 +713,8 @@ export type TypeTextOptions = DeviceCommandBaseOptions & {
697713

698714
export type FillOptions = DeviceCommandBaseOptions &
699715
SelectorSnapshotCommandOptions &
700-
InteractionTarget & {
716+
InteractionTarget &
717+
SettleCommandOptions & {
701718
text: string;
702719
delayMs?: number;
703720
verify?: boolean;
@@ -928,6 +945,8 @@ type CommandExecutionOptions = Partial<ScreenshotRequestFlags> & {
928945
pixels?: number;
929946
doubleTap?: boolean;
930947
verify?: boolean;
948+
settle?: boolean;
949+
settleQuietMs?: number;
931950
clickButton?: ClickButton;
932951
pauseMs?: number;
933952
pattern?: SwipePattern;

‎src/commands/cli-grammar/common.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,16 @@ export function selectorSnapshotOptionsFromFlags(flags: CliFlags): SelectorSnaps
8888
};
8989
}
9090

91+
// press/click/fill/longpress --settle (#1101). --timeout doubles as the settle
92+
// deadline on these commands (the daemon rejects it without --settle).
93+
export function settleInputFromFlags(flags: CliFlags): Record<string, unknown> {
94+
return compactRecord({
95+
settle: flags.settle,
96+
settleQuietMs: flags.settleQuietMs,
97+
timeoutMs: flags.timeoutMs,
98+
});
99+
}
100+
91101
export function repeatedInputFromFlags(flags: CliFlags): Record<string, unknown> {
92102
return compactRecord({
93103
count: flags.count,

‎src/commands/interaction/index.ts‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,11 @@ import type {
1818
TypeTextOptions,
1919
} from '../../client/client-types.ts';
2020
import type { CommandSchemaOverride } from '../../utils/cli-command-schema-types.ts';
21-
import { REPEATED_TOUCH_FLAGS, SELECTOR_SNAPSHOT_FLAGS } from '../../cli/parser/cli-flags.ts';
21+
import {
22+
REPEATED_TOUCH_FLAGS,
23+
SELECTOR_SNAPSHOT_FLAGS,
24+
SETTLE_FLAGS,
25+
} from '../../cli/parser/cli-flags.ts';
2226
import { defineCommandFacet, defineCommandFamilyFromFacets } from '../family/types.ts';
2327
import { defineExecutableCommand } from '../command-contract.ts';
2428
import {
@@ -70,23 +74,29 @@ const interactionCliSchemas = {
7074
usageOverride: 'click <x y|@ref|selector>',
7175
positionalArgs: ['target'],
7276
allowsExtraPositionals: true,
73-
allowedFlags: [...REPEATED_TOUCH_FLAGS, 'clickButton', 'verify', ...SELECTOR_SNAPSHOT_FLAGS],
77+
allowedFlags: [
78+
...REPEATED_TOUCH_FLAGS,
79+
'clickButton',
80+
'verify',
81+
...SETTLE_FLAGS,
82+
...SELECTOR_SNAPSHOT_FLAGS,
83+
],
7484
},
7585
press: {
7686
usageOverride: 'press <x y|@ref|selector>',
7787
helpDescription:
7888
'Short press a semantic UI target by ref, selector, or point. For native context menus or hold gestures, use longpress <target> <durationMs> instead of press --hold-ms.',
7989
positionalArgs: ['targetOrX', 'y?'],
8090
allowsExtraPositionals: true,
81-
allowedFlags: [...REPEATED_TOUCH_FLAGS, 'verify', ...SELECTOR_SNAPSHOT_FLAGS],
91+
allowedFlags: [...REPEATED_TOUCH_FLAGS, 'verify', ...SETTLE_FLAGS, ...SELECTOR_SNAPSHOT_FLAGS],
8292
},
8393
longpress: {
8494
usageOverride: 'longpress <x y|@ref|selector> [durationMs]',
8595
helpDescription:
8696
'Open native context menus or long-press targets by ref, selector, or point. Duration is positional, for example longpress @e12 800 or longpress 300 500 800.',
8797
positionalArgs: ['targetOrX', 'yOrDurationMs?', 'durationMs?'],
8898
allowsExtraPositionals: true,
89-
allowedFlags: [...SELECTOR_SNAPSHOT_FLAGS],
99+
allowedFlags: [...SETTLE_FLAGS, ...SELECTOR_SNAPSHOT_FLAGS],
90100
},
91101
swipe: {
92102
helpDescription: 'Swipe coordinates with optional repeat pattern',
@@ -114,7 +124,7 @@ const interactionCliSchemas = {
114124
usageOverride: 'fill <x> <y> <text> | fill <@ref|selector> <text>',
115125
positionalArgs: ['targetOrX', 'yOrText', 'text?'],
116126
allowsExtraPositionals: true,
117-
allowedFlags: [...SELECTOR_SNAPSHOT_FLAGS, 'delayMs', 'verify'],
127+
allowedFlags: [...SELECTOR_SNAPSHOT_FLAGS, 'delayMs', 'verify', ...SETTLE_FLAGS],
118128
},
119129
scroll: {
120130
usageOverride: 'scroll <direction|top|bottom> [amount] [--pixels <n>] [--duration-ms <ms>]',
@@ -343,6 +353,7 @@ function toClickOptions(input: ClickInput): ClickOptions {
343353
...toRepeatedOptions(input),
344354
button: input.button,
345355
verify: input.verify,
356+
...toSettleOptions(input),
346357
};
347358
}
348359

@@ -353,6 +364,7 @@ function toPressOptions(input: PressInput): PressOptions {
353364
...toSelectorSnapshotOptions(input),
354365
...toRepeatedOptions(input),
355366
verify: input.verify,
367+
...toSettleOptions(input),
356368
};
357369
}
358370

@@ -364,6 +376,7 @@ function toFillOptions(input: FillInput): FillOptions {
364376
text: input.text,
365377
delayMs: input.delayMs,
366378
verify: input.verify,
379+
...toSettleOptions(input),
367380
};
368381
}
369382

@@ -373,6 +386,19 @@ function toLongPressOptions(input: LongPressInput): LongPressOptions {
373386
...toClientInteractionTarget(input.target),
374387
...toSelectorSnapshotOptions(input),
375388
durationMs: input.durationMs,
389+
...toSettleOptions(input),
390+
};
391+
}
392+
393+
function toSettleOptions(input: {
394+
settle?: boolean;
395+
settleQuietMs?: number;
396+
timeoutMs?: number;
397+
}): Pick<PressOptions, 'settle' | 'settleQuietMs' | 'timeoutMs'> {
398+
return {
399+
settle: input.settle,
400+
settleQuietMs: input.settleQuietMs,
401+
timeoutMs: input.timeoutMs,
376402
};
377403
}
378404

‎src/commands/interaction/interactions.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ import {
2626
requiredDaemonString,
2727
repeatedInputFromFlags,
2828
selectorSnapshotInputFromFlags,
29+
settleInputFromFlags,
2930
targetInputFromClientTarget,
3031
} from '../cli-grammar/common.ts';
3132
import type { CliReader, DaemonWriter, CommandInput } from '../cli-grammar/types.ts';
@@ -35,6 +36,7 @@ export const interactionCliReaders = {
3536
...commonInputFromFlags(flags),
3637
...selectorSnapshotInputFromFlags(flags),
3738
...repeatedInputFromFlags(flags),
39+
...settleInputFromFlags(flags),
3840
target: targetInputFromClientTarget(readInteractionTargetFromPositionals(positionals)),
3941
button: flags.clickButton,
4042
verify: flags.verify,
@@ -43,6 +45,7 @@ export const interactionCliReaders = {
4345
...commonInputFromFlags(flags),
4446
...selectorSnapshotInputFromFlags(flags),
4547
...repeatedInputFromFlags(flags),
48+
...settleInputFromFlags(flags),
4649
target: targetInputFromClientTarget(readInteractionTargetFromPositionals(positionals)),
4750
verify: flags.verify,
4851
}),
@@ -51,6 +54,7 @@ export const interactionCliReaders = {
5154
return {
5255
...commonInputFromFlags(flags),
5356
...selectorSnapshotInputFromFlags(flags),
57+
...settleInputFromFlags(flags),
5458
target: targetInputFromClientTarget(decoded),
5559
durationMs: decoded.durationMs,
5660
};
@@ -79,6 +83,7 @@ export const interactionCliReaders = {
7983
return {
8084
...commonInputFromFlags(flags),
8185
...selectorSnapshotInputFromFlags(flags),
86+
...settleInputFromFlags(flags),
8287
target: targetInputFromClientTarget(decoded.target),
8388
text: decoded.text,
8489
delayMs: flags.delayMs,

‎src/commands/interaction/metadata.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,19 +71,29 @@ const verifyField = () =>
7171
'Capture cheap post-action evidence (AX digest, node counts, changedFromBefore) instead of a follow-up snapshot.',
7272
);
7373

74+
const settleFields = () => ({
75+
settle: booleanField(
76+
'After the action, wait for the UI to go quiet and return the settled diff vs the pre-action tree in the same response. Best-effort; never fails the action.',
77+
),
78+
settleQuietMs: integerField('Settle: quiet window in milliseconds (default 500).', { min: 0 }),
79+
timeoutMs: integerField('Settle: wait deadline in milliseconds (default 10000).', { min: 1 }),
80+
});
81+
7482
const clickFields = {
7583
target: requiredField(interactionTargetField()),
7684
button: enumField(CLICK_BUTTONS, 'Pointer button for platforms that support mouse buttons.'),
7785
...selectorSnapshotFields(),
7886
...repeatedFields(),
7987
verify: verifyField(),
88+
...settleFields(),
8089
};
8190

8291
const pressFields = {
8392
target: requiredField(interactionTargetField()),
8493
...selectorSnapshotFields(),
8594
...repeatedFields(),
8695
verify: verifyField(),
96+
...settleFields(),
8797
};
8898

8999
const fillFields = {
@@ -92,12 +102,14 @@ const fillFields = {
92102
delayMs: integerField('Delay between typed characters.', { min: 0 }),
93103
...selectorSnapshotFields(),
94104
verify: verifyField(),
105+
...settleFields(),
95106
};
96107

97108
const longPressFields = {
98109
target: requiredField(interactionTargetField()),
99110
durationMs: integerField('Long press duration in milliseconds.', { min: 0 }),
100111
...selectorSnapshotFields(),
112+
...settleFields(),
101113
};
102114

103115
const swipeFields = {

0 commit comments

Comments
 (0)