Skip to content

Commit 82fb8c2

Browse files
thymikeethymikee
committed
refactor: migrate type to the request-bound device runtime
Wave 5 unit 2 for #1739 (ADR 0019), find's last blocker. `type "text"` and `find <q> type "text"` now reach the device through one admitted, request-bound `typeText` operation instead of the `handleTypeCommand` interactor leaf and its dispatch-table arm. - New `TypeTextRuntimeOperations` contract riding the same `Interactor` seam as focus/screenshot/element-text; the operation returns the interactor's own closed `TypeTextBackendResult`, so Apple route evidence passes through and every other owner types blind, exactly as before. The iOS synthesized-type commit wait (#1676) is Apple-interactor-internal and moves nowhere. - The interaction backend's `typeText` member exists only when the `type` handler admitted and bound a runtime — no caller can fall back to legacy dispatch, so the command keeps exactly one execution path (R41). - `executeBoundTypeText` reproduces the retired leaf byte-for-byte: leading-ref rejection with the same hint, space-joined positionals, the 0-10000 delay bound, and only textEntryRoute surviving from the owner's result. Its parse pins moved from the dispatch-level tests into the daemon runtime test. - Exact-owner facts replace the capability bucket (apple sim+device, android all-but-simulator-row, harmonyos emulator+device, linux device, web device, vega unavailable, providers wherever their interactor is reachable) and `type` leaves HARMONYOS_SUPPORTED_COMMANDS / WEB_INTERACTION_COMMANDS. - The Linux desktop replay types a digit on real hardware; the coverage manifest promotes `type` contract -> live with the two-sided count pins. - Android/webdriver facts helpers extracted (androidTouchFact, interactorCell) to keep inspectFacts under the complexity gate. `find` stays legacy: both of its direct execution legs now share bound runtimes, so the atomic R35 cutover is next.
1 parent 07023eb commit 82fb8c2

41 files changed

Lines changed: 785 additions & 126 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/contracts/src/facades/platform.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,7 @@ export {
222222
snapshotRuntimePlanUses,
223223
waitSelectorCaptureRuntimePlanUses,
224224
focusRuntimeUse,
225+
typeTextRuntimeUse,
225226
viewportRuntimeUse,
226227
} from '../platform-runtime-operations.ts';
227228
export type {
@@ -301,6 +302,18 @@ export type {
301302
LocalFocusInteractorResolver,
302303
ProviderFocusInteractorResolver,
303304
} from '../focus-runtime.ts';
305+
export {
306+
bindLocalTypeTextInteractor,
307+
bindProviderTypeTextInteractor,
308+
typeTextRuntimeOperationFacts,
309+
} from '../type-text-runtime.ts';
310+
export type {
311+
LocalTypeTextInteractorResolver,
312+
ProviderTypeTextInteractorResolver,
313+
TypeTextInput,
314+
TypeTextRuntimeOperationFacts,
315+
TypeTextRuntimeOperations,
316+
} from '../type-text-runtime.ts';
304317
export { viewportRuntimeOperationFacts } from '../viewport-runtime.ts';
305318
export type {
306319
SetViewportInput,

‎packages/contracts/src/platform-runtime-operations.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import type { SnapshotRuntimeHost, SnapshotRuntimeOperations } from './snapshot-
1717
import type { SelectorObservationRuntimeOperations } from './selector-observation-runtime.ts';
1818
import type { ViewportRuntimeOperations } from './viewport-runtime.ts';
1919
import type { FocusRuntimeOperations } from './focus-runtime.ts';
20+
import type { TypeTextRuntimeOperations } from './type-text-runtime.ts';
2021
import type { ElementTextRuntimeOperations } from './element-text-runtime.ts';
2122
import type {
2223
DeviceReadinessRuntimeHost,
@@ -52,6 +53,7 @@ export type PlatformRuntimeOperations = AppLogRuntimeOperations &
5253
SelectorObservationRuntimeOperations &
5354
ViewportRuntimeOperations &
5455
FocusRuntimeOperations &
56+
TypeTextRuntimeOperations &
5557
ElementTextRuntimeOperations &
5658
DeviceReadinessRuntimeOperations &
5759
DeviceShutdownRuntimeOperations &
@@ -73,6 +75,7 @@ export const appsRuntimeUse = defineUse({ required: ['ensureReady', 'listApps']
7375
export const captureSnapshotUse = defineUse({ required: ['captureSnapshot'] });
7476
export const viewportRuntimeUse = defineUse({ required: ['setViewport'] });
7577
export const focusRuntimeUse = defineUse({ required: ['focusPoint'] });
78+
export const typeTextRuntimeUse = defineUse({ required: ['typeText'] });
7679
const captureSnapshotWithCustomActionsUse = defineUse({
7780
required: ['captureSnapshot', 'captureSnapshotWithCustomActions'],
7881
});

‎packages/contracts/src/platform-runtime-unavailable.test.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ test('generic unavailable binding preserves exact provider ownership and mode',
3232
screenshot: { available: false, reason: 'unsupported-device-kind' },
3333
viewport: { available: false, reason: 'unsupported-platform-leaf' },
3434
focus: { available: false, reason: 'unsupported-provider-mode' },
35+
typeText: { available: false, reason: 'unsupported-provider-mode' },
3536
elementText: { available: false, reason: 'unsupported-provider-mode' },
3637
lifecycle,
3738
});
@@ -47,6 +48,10 @@ test('generic unavailable binding preserves exact provider ownership and mode',
4748
available: false,
4849
reason: 'unsupported-provider-mode',
4950
});
51+
assert.deepEqual(binding.facts.operations.typeText, {
52+
available: false,
53+
reason: 'unsupported-provider-mode',
54+
});
5055
// Both capture cells are owner-stated, so neither inherits the network gap's reason.
5156
assert.deepEqual(binding.facts.operations.captureScreenshot, {
5257
available: false,

‎packages/contracts/src/platform-runtime-unavailable.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import { snapshotRuntimeOperationFacts } from './snapshot-runtime.ts';
1515
import { selectorObservationRuntimeOperationFacts } from './selector-observation-runtime.ts';
1616
import { viewportRuntimeOperationFacts } from './viewport-runtime.ts';
1717
import { focusRuntimeOperationFacts } from './focus-runtime.ts';
18+
import { typeTextRuntimeOperationFacts } from './type-text-runtime.ts';
1819
import { elementTextRuntimeOperationFacts } from './element-text-runtime.ts';
1920

2021
/**
@@ -32,6 +33,7 @@ export type UnavailablePlatformRuntimeFacts = Readonly<{
3233
snapshot?: RuntimeOperationUnavailability;
3334
viewport: RuntimeOperationUnavailability;
3435
focus: RuntimeOperationUnavailability;
36+
typeText: RuntimeOperationUnavailability;
3537
elementText: RuntimeOperationUnavailability;
3638
readiness?: RuntimeOperationUnavailability;
3739
shutdown?: RuntimeOperationUnavailability;
@@ -77,6 +79,7 @@ export function createUnavailablePlatformRuntimeFacts(
7779
snapshot,
7880
viewport,
7981
focus,
82+
typeText,
8083
elementText,
8184
readiness,
8285
shutdown,
@@ -118,6 +121,7 @@ export function createUnavailablePlatformRuntimeFacts(
118121
}),
119122
...viewportRuntimeOperationFacts({ setViewport: viewport }),
120123
...focusRuntimeOperationFacts({ focus }),
124+
...typeTextRuntimeOperationFacts({ type: typeText }),
121125
...elementTextRuntimeOperationFacts({ readTextAtPoint: elementText }),
122126
ensureReady: readiness,
123127
bootTarget: readiness,
@@ -149,6 +153,7 @@ function freezeUnavailableFacts(
149153
// Interaction cells are stated by their owner: a family that can drive touch says so for its
150154
// exact kinds, and one that cannot must say why rather than inherit a transport gap.
151155
focus: Object.freeze({ ...unavailable.focus }),
156+
typeText: Object.freeze({ ...unavailable.typeText }),
152157
readiness: orNetwork(unavailable.readiness),
153158
shutdown: orNetwork(unavailable.shutdown),
154159
elementText: Object.freeze({ ...unavailable.elementText }),

‎packages/contracts/src/type-text-runtime.test.ts‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
import { expect, test, vi } from 'vitest';
2+
import type { Interactor } from './interactor-types.ts';
3+
import {
4+
bindLocalTypeTextInteractor,
5+
bindProviderTypeTextInteractor,
6+
typeTextRuntimeOperationFacts,
7+
} from './type-text-runtime.ts';
8+
9+
const device = {
10+
platform: 'android',
11+
id: 'emulator-5554',
12+
name: 'Pixel',
13+
kind: 'emulator',
14+
booted: true,
15+
} as const;
16+
17+
test('builds the exact type operation fact catalog', () => {
18+
const type = { available: true } as const;
19+
expect(typeTextRuntimeOperationFacts({ type })).toEqual({ typeText: type });
20+
});
21+
22+
test('a local binding drives the interactor with the text, delay, and resolver context', async () => {
23+
const type = vi.fn(async () => ({ textEntryRoute: 'synthesized-first-responder' }) as const);
24+
const resolveInteractor = vi.fn(async () => ({ type }) as unknown as Interactor);
25+
const signal = new AbortController().signal;
26+
27+
const operations = bindLocalTypeTextInteractor({ device, signal, resolveInteractor });
28+
const result = await operations.typeText({
29+
text: 'hello world',
30+
delayMs: 25,
31+
options: { appBundleId: 'com.example.app' },
32+
execution: { logPath: '/tmp/daemon.log', requestId: 'type-1' },
33+
});
34+
35+
expect(resolveInteractor).toHaveBeenCalledWith(device, {
36+
logPath: '/tmp/daemon.log',
37+
requestId: 'type-1',
38+
appBundleId: 'com.example.app',
39+
signal,
40+
});
41+
// Positional (text, delayMs): the `Interactor` seam takes two arguments, and a swapped or
42+
// dropped delay is the drift a structural assertion would not catch.
43+
expect(type).toHaveBeenCalledWith('hello world', 25);
44+
// The owner's closed result passes through untouched — the route evidence is the payload.
45+
expect(result).toEqual({ textEntryRoute: 'synthesized-first-responder' });
46+
});
47+
48+
test('a provider binding drives its own resolved interactor and passes a blind result through', async () => {
49+
const type = vi.fn(async () => undefined);
50+
const resolveInteractor = vi.fn(() => ({ type }) as unknown as Interactor);
51+
const signal = new AbortController().signal;
52+
53+
const operations = bindProviderTypeTextInteractor({ device, signal, resolveInteractor });
54+
const result = await operations.typeText({ text: 'abc', delayMs: 0 });
55+
56+
expect(resolveInteractor).toHaveBeenCalledWith({
57+
appBundleId: undefined,
58+
signal,
59+
});
60+
expect(type).toHaveBeenCalledWith('abc', 0);
61+
expect(result).toBeUndefined();
62+
});
63+
64+
test('a provider binding fails closed when its exact owner exposes no interactor', async () => {
65+
const operations = bindProviderTypeTextInteractor({
66+
device,
67+
signal: new AbortController().signal,
68+
resolveInteractor: () => undefined,
69+
});
70+
71+
await expect(operations.typeText({ text: 'abc', delayMs: 0 })).rejects.toMatchObject({
72+
code: 'UNSUPPORTED_OPERATION',
73+
details: { reason: 'provider-runtime-interactor-missing', deviceId: device.id },
74+
});
75+
});
76+
77+
test('an already-cancelled request never resolves an interactor', async () => {
78+
const controller = new AbortController();
79+
controller.abort();
80+
const type = vi.fn(async () => undefined);
81+
const resolveInteractor = vi.fn(async () => ({ type }) as unknown as Interactor);
82+
83+
const operations = bindLocalTypeTextInteractor({
84+
device,
85+
signal: controller.signal,
86+
resolveInteractor,
87+
});
88+
89+
await expect(operations.typeText({ text: 'abc', delayMs: 0 })).rejects.toThrow();
90+
expect(resolveInteractor).not.toHaveBeenCalled();
91+
expect(type).not.toHaveBeenCalled();
92+
});

‎packages/contracts/src/type-text-runtime.ts‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
import type { DeviceInfo } from '@agent-device/kernel/device';
2+
import { AppError } from '@agent-device/kernel/errors';
3+
import type { Interactor, RunnerContext, TypeTextBackendResult } from './interactor-types.ts';
4+
import type { RuntimeOperationFact } from './platform-runtime.ts';
5+
import type { SessionSurface } from './session-surface.ts';
6+
import type { SnapshotRuntimeExecution } from './snapshot-runtime.ts';
7+
8+
/**
9+
* Neutral intent for one focus-target text entry. The text goes wherever the platform's current
10+
* first responder is — targeting happened before this operation (a `focus`/`press`, or find's
11+
* resolution) — so the operation names no command, request, session, or CLI flag. `delayMs` is
12+
* already validated by the caller; the owner receives it as data.
13+
*/
14+
export type TypeTextInput = Readonly<{
15+
text: string;
16+
delayMs: number;
17+
options?: Readonly<{ appBundleId?: string; surface?: SessionSurface }>;
18+
/** Same runner metadata a capture needs; reuses that type rather than restating it. */
19+
execution?: SnapshotRuntimeExecution;
20+
}>;
21+
22+
/**
23+
* The operation returns the interactor's own closed result: only the Apple runner reports which
24+
* text-entry route it took; every other owner types blind and returns nothing (`interactor-types`
25+
* owns that contract and its Swift route-parity test).
26+
*/
27+
export type TypeTextRuntimeOperations = Readonly<{
28+
typeText(input: TypeTextInput): Promise<TypeTextBackendResult | void>;
29+
}>;
30+
31+
export type TypeTextRuntimeOperationFacts = Readonly<{
32+
typeText: RuntimeOperationFact;
33+
}>;
34+
35+
export function typeTextRuntimeOperationFacts(
36+
input: Readonly<{ type: RuntimeOperationFact }>,
37+
): TypeTextRuntimeOperationFacts {
38+
return Object.freeze({ typeText: input.type });
39+
}
40+
41+
/**
42+
* Captures one selected owner's interactor authority for the lifetime of a request binding. The
43+
* owner is already chosen by the time a binder is called, so each entry point supplies its own
44+
* resolution and this holds only what both share: the runner context and the entry itself.
45+
*/
46+
function bindTypeText(
47+
signal: AbortSignal,
48+
resolveInteractor: (runner: RunnerContext) => Promise<Interactor>,
49+
): TypeTextRuntimeOperations {
50+
return Object.freeze({
51+
typeText: async (input: TypeTextInput) => {
52+
signal.throwIfAborted();
53+
const interactor = await resolveInteractor({
54+
...input.execution,
55+
appBundleId: input.options?.appBundleId,
56+
signal,
57+
});
58+
return await interactor.type(input.text, input.delayMs);
59+
},
60+
});
61+
}
62+
63+
export type LocalTypeTextInteractorResolver = (
64+
device: DeviceInfo,
65+
runner: RunnerContext,
66+
) => Promise<Interactor>;
67+
68+
export function bindLocalTypeTextInteractor(
69+
params: Readonly<{
70+
device: DeviceInfo;
71+
signal: AbortSignal;
72+
resolveInteractor: LocalTypeTextInteractorResolver;
73+
}>,
74+
): TypeTextRuntimeOperations {
75+
return bindTypeText(
76+
params.signal,
77+
async (runner) => await params.resolveInteractor(params.device, runner),
78+
);
79+
}
80+
81+
export type ProviderTypeTextInteractorResolver = (runner: RunnerContext) => Interactor | undefined;
82+
83+
/** Provider bindings fail closed when their exact owner no longer exposes its interactor. */
84+
export function bindProviderTypeTextInteractor(
85+
params: Readonly<{
86+
device: DeviceInfo;
87+
signal: AbortSignal;
88+
resolveInteractor: ProviderTypeTextInteractorResolver;
89+
}>,
90+
): TypeTextRuntimeOperations {
91+
return bindTypeText(params.signal, async (runner) => {
92+
const interactor = params.resolveInteractor(runner);
93+
if (interactor) return interactor;
94+
throw new AppError(
95+
'UNSUPPORTED_OPERATION',
96+
'Provider-owned type operation has no bound provider interactor.',
97+
{ reason: 'provider-runtime-interactor-missing', deviceId: params.device.id },
98+
);
99+
});
100+
}

‎packages/platform-android/src/runtime.ts‎

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,12 @@ import {
1111
availableApplicationLifecycleOperations,
1212
bindLocalFocusInteractor,
1313
bindLocalScreenshotInteractor,
14+
bindLocalTypeTextInteractor,
1415
bindElementTextRuntime,
1516
bindLocalSnapshotInteractor,
1617
elementTextRuntimeOperationFacts,
1718
focusRuntimeOperationFacts,
19+
typeTextRuntimeOperationFacts,
1820
localRuntimeOwner,
1921
screenshotRuntimeOperationFacts,
2022
selectorObservationRuntimeOperationFacts,
@@ -142,6 +144,11 @@ function androidRuntimeHintsFact(device: DeviceInfo) {
142144
: runtimeHintsUnavailable;
143145
}
144146

147+
/** adb drives every interaction cell the same way; only the synthetic `simulator` row lacks a device. */
148+
function androidTouchFact(device: DeviceInfo) {
149+
return device.kind === 'simulator' ? focusKindUnavailable : available;
150+
}
151+
145152
export function createAndroidPlatformRuntime(host: PlatformRuntimeHost): PlatformRuntimeOwner {
146153
const appLogs = createAndroidAppLogRuntime(host);
147154
const inspectFacts = async (device: Parameters<typeof appLogs.inspectFacts>[0]) => {
@@ -171,9 +178,10 @@ export function createAndroidPlatformRuntime(host: PlatformRuntimeHost): Platfor
171178
findSelector: snapshotKindUnavailable,
172179
}),
173180
...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }),
174-
...focusRuntimeOperationFacts({
175-
focus: device.kind === 'simulator' ? focusKindUnavailable : available,
176-
}),
181+
...focusRuntimeOperationFacts({ focus: androidTouchFact(device) }),
182+
// Text entry shares focus's cell: adb drives both, and only the synthetic `simulator`
183+
// row has no device behind it (parity with the retired `type` bucket).
184+
...typeTextRuntimeOperationFacts({ type: androidTouchFact(device) }),
177185
// uiautomator reads text at a point through the same adb path the snapshot uses, so the
178186
// synthetic `simulator` row is the only Android kind without a live read.
179187
...elementTextRuntimeOperationFacts({
@@ -246,6 +254,13 @@ export function createAndroidPlatformRuntime(host: PlatformRuntimeHost): Platfor
246254
resolveInteractor: host.localInteractors.resolve,
247255
})
248256
: {}),
257+
...(facts.operations.typeText.available
258+
? bindLocalTypeTextInteractor({
259+
device: request.device,
260+
signal: request.scope.signal,
261+
resolveInteractor: host.localInteractors.resolve,
262+
})
263+
: {}),
249264
...(facts.operations.readTextAtPoint.available
250265
? bindElementTextRuntime({
251266
device: request.device,

‎packages/platform-apple/src/runtime.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,11 @@ import {
1111
availableApplicationLifecycleOperations,
1212
bindLocalFocusInteractor,
1313
bindLocalScreenshotInteractor,
14+
bindLocalTypeTextInteractor,
1415
bindElementTextRuntime,
1516
elementTextRuntimeOperationFacts,
1617
focusRuntimeOperationFacts,
18+
typeTextRuntimeOperationFacts,
1719
localRuntimeOwner,
1820
screenshotRuntimeOperationFacts,
1921
selectorObservationRuntimeOperationFacts,
@@ -267,6 +269,9 @@ export function createApplePlatformRuntime(host: PlatformRuntimeHost): PlatformR
267269
}),
268270
...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }),
269271
...focusRuntimeOperationFacts({ focus: appleFocusFact(device) }),
272+
// Text entry rides the same interactor authority the point focus does, so it shares the
273+
// exact kind cell (parity with the retired `type` bucket, `{ simulator, device }`).
274+
...typeTextRuntimeOperationFacts({ type: appleFocusFact(device) }),
270275
...elementTextRuntimeOperationFacts({ readTextAtPoint: appleElementTextFact(device) }),
271276
ensureReady: readiness,
272277
bootTarget: boot,
@@ -326,6 +331,13 @@ export function createApplePlatformRuntime(host: PlatformRuntimeHost): PlatformR
326331
resolveInteractor: host.localInteractors.resolve,
327332
}),
328333
),
334+
...whenAdmitted(facts.operations.typeText, () =>
335+
bindLocalTypeTextInteractor({
336+
device: request.device,
337+
signal: request.scope.signal,
338+
resolveInteractor: host.localInteractors.resolve,
339+
}),
340+
),
329341
...whenAdmitted(facts.operations.readTextAtPoint, () =>
330342
bindElementTextRuntime({
331343
device: request.device,

0 commit comments

Comments
 (0)