Skip to content

Commit 4b8bcac

Browse files
feat(interaction): accept fill <target> "" as the clear-field primitive (callstack#2066)
* feat(interaction): accept fill <target> "" as the clear-field primitive Emptying an input was not expressible: `fill` refused the empty string ("Expected text to be a non-empty string"), `type` only appends, and `keyboard` has no delete verb. Clearing a field before typing is a routine QA step, so the only route was the app's own clear button or N locale-dependent keyboard delete presses read out of a snapshot. `fill <target> ""` now means "replace with nothing". Both platforms already own the clear half of replace, so this is the validation and reporting that stood in front of it, not a new interaction: - `stringField` takes an opt-in `allowEmpty`, used only by `fill`'s `text`. `requiredField` still refuses a MISSING text, so `fill @e57` stays an error rather than silently erasing the field — `readFillTargetFromPositionals` now reports `undefined` for "no text argument" instead of collapsing it to `''`. `type` keeps refusing an empty text: appending nothing is not a clear. - The Apple runner's empty-text early return skipped the clear while reporting "typed". For a replacement it now runs `clearTextInput` and verifies the field came back empty (secure fields stay unverifiable, as elsewhere). - Android already clears before typing and skips an empty shell/IME write, but its verifier read a cleared field's absent `text` attribute as a mismatch against `''`. An empty expectation now accepts null or "". Whitespace-only text keeps its established per-shape rules; only `''` is new. Closes callstack#2063 * fix(interaction): fail the empty-fill clear closed on every backend Addresses the P1 review on callstack#2066, then closes the same fail-open class on the backends the PR did not reach: - Android: an empty expectation no longer matches when the verification scan observed NO input node at all — actual is null both for a cleared field and for a wrong point/lost focus, and three empty samples of nothing were a stable success for a clear that never touched a field. - Apple runner: when the empty-replacement path cannot resolve a clear target (including the synthesized first-responder route, whose target carries no element), it returns the typed TEXT_INPUT_NOT_FOCUSED failure instead of falling through to the vacuous-typing verified-success return. Regression runs in the ios.yml XCTest lane. - webdriver: fill is tap + sendKeys and owns no clear mechanism, so an empty fill refuses as UNSUPPORTED_OPERATION before touching the device, instead of reporting a clear it cannot perform. - linux + web coordinate fill: typing zero characters over the select-all selection left the old value intact; the empty fill now deletes the selection. - recording: an empty --record-as literal matches inside every string; it now parameterizes only the fill's own text field instead of rewriting every empty field and empty evidence label in the entry. (The session-wide echo registry already excluded empty literals.) - help: the text-entry topic taught agents that fill "" is not a clear-field command; it now states the new contract. Each new test was observed red against the pre-fix code. * fix(android): read hint-showing from the helper so a cleared field verifies Live Pixel 9 emulator, adb-shell channel: clearing the Settings search field succeeded on the device but reported 'Android fill verification failed', because a cleared EditText dumps its HINT as text — getText() returns the hint for an empty field on modern Android, so 'Search settings' read back as a residual value. This is the same placeholder-as-value trap the Apple runner already handles with treatingPlaceholderAsEmpty. The helper now emits hint-showing (isShowingHintText, API 26+), the hierarchy parser carries it, and fill verification matches against the field's VALUE — hint-only text is an empty value, for empty and non-empty expectations alike. A field whose real value equals its hint string keeps failing the clear check: only the authoritative flag, never the text, says it is a hint. Raw uiautomator dumps carry no such fact and keep the fail-closed behavior. Live evidence, both admission channels, after this fix: test-ime and adb-shell clears both report Filled 0 chars with the field back on its placeholder; the pre-fix adb-shell run failed closed (never a false success). * fix(interaction): close the adversarial-review findings on the empty-fill clear - android adb-shell: the delete burst is sized from the value being REMOVED (pre-mutation read; the attempt's cap when unreadable), not from the empty incoming text, which sent the 12/24-delete minimums and could never empty a field longer than 36 characters. - android: the unconfirmed soft-success no longer applies to an empty expectation — nothing app-formats the empty value, so residue after a clear is a failed clear, and the soft-success also skipped the second, bigger delete burst. - android masked fields: an empty expectation accepts an observed masked node with no dump text (a masked field WITH content dumps its bullet run), so clearing a password field no longer fails after the clear worked — matching iOS, where a secure-field clear succeeds unverified. - find: 'find <q> fill ""' now reaches the fill leaf as the clear request on both the CLI reader and the daemon positional parse; a MISSING value keeps its refusal at each producer, so the typed value: string contract is unchanged. - maestro export: a recorded clear exports as tapOn + eraseText instead of a vacuous inputText: "" (with the 50-character-default warning). - the missing-text refusals teach the clear form: (use "" to clear the field). Full unit suite green (1061 files); each behavioral fix carries a test observed red against the prior code. * refactor(interaction,android): extract the fill parse and shell-attempt branches The review commits pushed parseFillTarget and fillAndroid over the complexity gate (13 cyclomatic each). Each fill target shape parses in its own function sharing one missing-text response, and the adb-shell attempt (clear sizing + clear + type + verify) moves out of the fill loop. Behavior-preserving; the existing tests cover every branch. * refactor(interaction,android): one owner per empty-fill fact Design pass after review: the missing-vs-empty rule and the observed- value rule each had several owners; now each has one. - parseFillTarget decodes ONCE through readFillTargetFromPositionals — which already owns shape detection and documents the undefined-vs-'' contract on DecodedFillTarget — and keeps only what the wire owns: versioned-ref admission, the selector whitespace rule, and the daemon responses. This deletes the point branch's duplicated slicing, the hasFillText guard, and the three per-shape parse functions. - observedAndroidValue() is the single statement of Android's value rule (absent attribute and hint-only text are the empty value); the text branch, the match rule, and the masked branch all consume it. The masked branch thereby gains the hint-showing collapse it was missing, and isAcceptableAndroidFillMatch narrows to plain strings. - The empty-text-is-clear contract is stated once, on Interactor.fill in contracts, instead of implied per backend. Behavior-preserving except the masked+hint gain; the existing tests cover every branch (494 Android, 15 fill-target). --------- Co-authored-by: Michał Pierzchała <[email protected]>
1 parent edba835 commit 4b8bcac

34 files changed

Lines changed: 749 additions & 126 deletions

File tree

‎.github/workflows/ios.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,7 @@ jobs:
144144
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testPenalizedCoordinateTapOnNonTextControlDoesNotAuthorizeBareType \
145145
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testBareDelayedTypeFailsWhenTappedInputDisappearsMidCommand \
146146
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testSynthesizedTextCommitProgressWalksExpectedPrefixOnly \
147+
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testEmptyReplacementWithoutResolvableTargetFailsClosed \
147148
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testTextEntryTapWitnessIsBoundToTargetIdentity \
148149
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testCoordinateTapTextInputProbeSkipsPenalizedXCTestChannel \
149150
-only-testing:AgentDeviceRunnerUITests/RunnerTests/testCoordinateTextInputCandidateMustBeEnabledAndContainTheTouchPoint \

‎android/snapshot-helper/src/main/java/com/callstack/agentdevice/snapshothelper/AccessibilityTreeXml.java‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,13 @@ static void appendNode(
3535
appendWindowMetadata(xml, windowMetadata);
3636
}
3737
appendNonEmptyAttribute(xml, "text", node.getText());
38+
// getText() returns the HINT for an empty field on modern Android, so `text` alone cannot
39+
// distinguish a cleared field from one whose value equals its hint; only this flag can
40+
// (#2063 empty-fill verification).
41+
appendTrueAttribute(
42+
xml,
43+
"hint-showing",
44+
Build.VERSION.SDK_INT >= Build.VERSION_CODES.O && node.isShowingHintText());
3845
appendNonEmptyAttribute(xml, "resource-id", node.getViewIdResourceName());
3946
appendAttribute(xml, "class", node.getClassName());
4047
appendNonEmptyAttribute(xml, "package", node.getPackageName());

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextTyping.swift‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,36 @@ extension RunnerTests {
1515
) -> TextEntryResult {
1616
let totalStartedAt = Date()
1717
guard !text.isEmpty else {
18+
// Replacing a field WITH the empty string is the clear-field primitive (#2063): the typing
19+
// is vacuous but the clear is not. Returning here unconditionally left the old value in
20+
// place while the response reported "typed". Append (`type`) and unrepaired entry stay
21+
// no-ops, which is what an empty payload means for them.
22+
if repairMode == .replacement {
23+
guard let clearTarget = resolveTextEntryElement(app: app, target: target) else {
24+
// No resolvable input means no clear happened — including the synthesized
25+
// first-responder route, whose target carries no element. Success here would claim a
26+
// clear the runner never performed.
27+
logTextEntryPhase(commandId: commandId, phase: "total", startedAt: totalStartedAt, chars: 0, mode: repairMode)
28+
return TextEntryResult(
29+
verified: nil,
30+
repaired: false,
31+
expectedText: "",
32+
observedText: nil,
33+
failure: .notFocused
34+
)
35+
}
36+
clearTextInput(clearTarget)
37+
// nil means the value is unreadable (secure fields), not "not empty": leave `verified`
38+
// nil there rather than reporting a mismatch the runner cannot actually observe.
39+
let observed = editableTextValue(for: clearTarget, treatingPlaceholderAsEmpty: true)
40+
logTextEntryPhase(commandId: commandId, phase: "total", startedAt: totalStartedAt, chars: 0, mode: repairMode)
41+
return TextEntryResult(
42+
verified: observed.map { $0.isEmpty },
43+
repaired: false,
44+
expectedText: "",
45+
observedText: observed
46+
)
47+
}
1848
logTextEntryPhase(commandId: commandId, phase: "total", startedAt: totalStartedAt, chars: 0, mode: repairMode)
1949
return TextEntryResult(verified: true, repaired: false, expectedText: "", observedText: "")
2050
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+TextEntryPolicyTests.swift‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -445,6 +445,26 @@ extension RunnerTests {
445445
XCTAssertEqual(result.failure, .commitNotObserved)
446446
}
447447

448+
// `fill <target> ""` is the clear-field primitive (#2063). When no clear target resolves —
449+
// Springboard's home screen has no focused text input — the empty-text replacement path must
450+
// fail closed: it used to fall through to the vacuous-typing early return and report
451+
// `verified: true` for a clear that never ran.
452+
func testEmptyReplacementWithoutResolvableTargetFailsClosed() {
453+
let result = typeTextReliably(
454+
app: springboard,
455+
target: TextEntryTarget(element: nil, refreshPoint: nil, prefersFocusedElement: false),
456+
text: "",
457+
delaySeconds: 0,
458+
repairMode: .replacement,
459+
xCTestChannelPenalized: false,
460+
synthesizer: RecordingTextEntrySynthesizer()
461+
)
462+
463+
XCTAssertEqual(result.failure, .notFocused)
464+
XCTAssertNil(result.verified)
465+
XCTAssertNil(result.observedText)
466+
}
467+
448468
// Companion to the above: text carrying a submit key must skip the wait entirely, same as the
449469
// append route (`awaitSynthesizedFirstResponderCommit`) — the app may clear or rewrite the field
450470
// on submit, so there is nothing meaningful to poll toward.

‎packages/contracts/src/interactor-types.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,6 +285,12 @@ export type Interactor = {
285285
hoverRef?(ref: string): Promise<Record<string, unknown> | void>;
286286
focus(x: number, y: number): Promise<Record<string, unknown> | void>;
287287
type(text: string, delayMs?: number): Promise<TypeTextBackendResult | void>;
288+
/**
289+
* Replace the target's text with `text`. The empty string is the clear request (#2063), not a
290+
* no-op: an implementation must empty the field or fail — never report success over an
291+
* untouched value. A backend with no clear mechanism refuses the empty text up front (see the
292+
* webdriver interactor).
293+
*/
288294
fill(
289295
x: number,
290296
y: number,

‎packages/maestro/src/internal/export-flow.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,20 @@ function convertFillAction(
266266
}
267267
const tapTarget = readTapTarget(target, undefined, resolveSelector);
268268
if (!tapTarget) return { kind: 'unsupported', message: 'fill target is not Maestro-compatible' };
269+
if (text.length === 0) {
270+
// The clear request (#2063): `inputText: ""` types nothing in Maestro, so the recorded
271+
// clear would silently become a no-op. `eraseText` is Maestro's clear verb; without a
272+
// count it erases up to its 50-character default, which is a bound this export cannot
273+
// recover the real length for.
274+
return {
275+
kind: 'commands',
276+
commands: [{ tapOn: tapTarget }, 'eraseText'],
277+
warnings: [
278+
...readLabelSelectorWarnings(target),
279+
'fill "" exports as tapOn + eraseText; Maestro erases at most its 50-character default',
280+
],
281+
};
282+
}
269283
return {
270284
kind: 'commands',
271285
commands: [{ tapOn: tapTarget }, { inputText: text }],

‎packages/provider-webdriver/src/webdriver-interactor.test.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,27 @@ test('fill caps each probe timeout by the readiness budget remaining', async ()
228228
* past the whole readiness budget is safe for every case — once readiness
229229
* resolves there are no pending timers left to fire.
230230
*/
231+
// `fill <target> ""` is the clear-field primitive (#2063). This backend's fill is tap +
232+
// sendKeys and owns no clear mechanism, so an empty fill must refuse up front — sending zero
233+
// keys and reporting success would claim a clear that never ran, before even touching the
234+
// device.
235+
test('fill refuses empty text as an unsupported clear rather than a vacuous success', async () => {
236+
const world = createTextEntryWorld();
237+
const interactor = createWebDriverInteractor({
238+
client: world.client,
239+
backend: 'xctest',
240+
capabilities: createCloudWebDriverCapabilities({ provider: 'test', platform: 'ios' }),
241+
});
242+
243+
await assert.rejects(interactor.fill(12, 24, ''), (error: AppError) => {
244+
assert.equal(error.code, 'UNSUPPORTED_OPERATION');
245+
assert.match(error.message, /clear field/);
246+
return true;
247+
});
248+
// Fail-closed means untouched: no tap, no keys, no probes reached the device.
249+
assert.deepEqual(world.transcript, []);
250+
});
251+
231252
async function runFill(world: ReturnType<typeof createTextEntryWorld>) {
232253
vi.useFakeTimers();
233254
try {

‎packages/provider-webdriver/src/webdriver-interactor.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,6 +215,18 @@ class WebDriverInteractor implements Interactor {
215215
_delayMs?: number,
216216
): Promise<Record<string, unknown>> {
217217
this.requireSupport('fill');
218+
// This backend's fill is tap + sendKeys: it never clears the existing value, so the
219+
// clear-field request (`fill <target> ""`, #2063) has no mechanism here — sending zero keys
220+
// and reporting success would claim a clear that never ran.
221+
if (text.length === 0) {
222+
throw new AppError(
223+
'UNSUPPORTED_OPERATION',
224+
'fill with empty text (clear field) is not supported on the webdriver backend',
225+
{
226+
hint: 'Use a visible clear/reset control in the app, or fill the field with the intended replacement text directly.',
227+
},
228+
);
229+
}
218230
// #1658: both readings are taken BEFORE the tap, because each is only
219231
// evidence as a change. A keyboard that was already up says nothing about
220232
// which field owns first responder now, and a focused element is only

‎packages/selectors/src/internal/find.ts‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,15 @@ export function parseFindArgs(args: string[]): ParsedFindArgs {
234234
return { locator, query, action: 'wait', timeoutMs: timeoutMs ?? undefined };
235235
}
236236
if (action === 'fill' || action === 'type') {
237-
return { locator, query, action, value: actionTokens.slice(1).join(' ') };
237+
// `undefined` when NO value token followed, `''` when an empty one did: `find <q> fill ""`
238+
// is the clear request (#2063) and must stay distinguishable from a forgotten argument.
239+
const valueTokens = actionTokens.slice(1);
240+
return {
241+
locator,
242+
query,
243+
action,
244+
value: valueTokens.length === 0 ? undefined : valueTokens.join(' '),
245+
};
238246
}
239247
throw new AppError('INVALID_ARGS', `Unsupported find action: ${actionTokens[0]}`, {
240248
hint: UNSUPPORTED_FIND_ACTION_HINT,

‎src/cli-schema/cli-help-topics.test.ts‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -159,10 +159,7 @@ test('usageForCommand resolves workflow help topic', async () => {
159159
assert.match(help, /geometry never chooses a winner/);
160160
assert.match(help, /iOS AX flags are unreliable on deep RN trees/);
161161
assert.match(help, /targetHittable: false plus a hint -- verify or re-target, not a failure/);
162-
assert.match(
163-
help,
164-
/Empty replacement is not a clear-field command \(do not plan fill <target> ""\)/,
165-
);
162+
assert.match(help, /fill <target> "" clears the field \(replace with nothing\)/);
166163
assert.match(help, /retry with --delay-ms before clipboard paste/);
167164
assert.match(
168165
help,

0 commit comments

Comments
 (0)