Skip to content

Commit d71441e

Browse files
authored
fix(ios): observe fill once without destructive repair (#3334)
* fix(ios): stop fill from failing and repairing fields that normalize their input (#2634) The bug was textEntryValueEchoes' containment clauses, not the repair gate. A field that reformats what it holds (a '## ### ## ##' digit grouping, a phone mask, a currency format) shows the request with the formatter's separators inserted; clauses 2/3 classify that as an echo, which withheld the unconfirmed outcome, so the fill fell to mismatch -> destructive clear-and-retype of a correct field -> TEXT_ENTRY_MISMATCH. Measured on iPhone 18 Pro / iOS 27.0 against a digit-grouping fixture: REPAIR_TEXT_ENTRY expectedLength=9 observedLength=12, then the mismatch. A value that COMPLETES the request now falls through to the existing unconfirmed-evidence path: every request character in order, plus inserted characters strictly between the request's own characters that appear nowhere in the request. Degradation removes; entry cannot insert a separator between two characters the same burst typed, and a failed clear leaves residual text at the ENDS of the typed run — so end-only insertions ('old123456') stay echoes and keep their repair, while an interior foreign insertion does not. Leftmost embedding and a no-request-character-extras guard keep doubled entries and ambiguous embeddings on the failure side. This deliberately flips the pin that '(555) 123-4567' IS an echo: 'echo' means 'could be a degraded copy', and a complete mask value is not one. Completion is never a correctness claim — a cents-shifting mask passes it — so the outcome is unconfirmed with before/after evidence, never verified: true; only exact equality verifies. fill-evidence.ts already names app-owned formatting as this shape's purpose; Android already reports it unconfirmed; iOS was the outlier. isRepairableTextEntryMismatch and the synthesized-replacement commit wait are UNCHANGED; the coordinate route's exact-match-only settle is pinned as an explicit non-goal. The write-back-truncation shape (ada@example -> adxe) stays a typed failure under #2903's ownership, and so do drops, stale residuals, and doubled entries. Fixture: AgentDeviceDigitGroupingTextField reproduces the reported formatter and restores the caret across reformats the way the real formatter it models does; both normalizing device tests share one fill-and-assert helper so their plumbing cannot drift. The device test proves the repair no longer runs (message 'typed', not 'typed after repair') and the field keeps 00 062 91 77. * test(ios): pin the completion rule's residual provenance on both sides (#2634 review) Adds the reviewer-requested classifier-table pair: baseline "12" refuses "12 567 8" for request "5678", while an absorbed residual (baseline "12", observed "(555) 123-4567") completes because every extra is punctuation the baseline cannot explain. The evidence level gains the same positive, so the rule is pinned as keyed on extra provenance, not baseline emptiness. Tightens the completion predicate's doc to match. * docs+test(ios): pin the template-retaining mask as the completion rule's known limit (#2634 review) The docs and help claimed a normalizing field is always typed once; a mask that keeps its template through the clear ("$0.00" before "$10.00") shares characters with the value it formats, so its fill still takes the echo reading with its repair — text cannot tell surviving template from failed-clear residual, which is the same relation the residual fix relies on. Pin the exact triple at the completion-table and evidence levels, state the boundary in the docs/help and the predicate doc, and drop the orphaned one-character-request comment left above the span bounds when the count guard moved into the opening guard. * fix(ios): observe fill once without destructive repair * fix(ios): preserve fill settling after slow reads * docs+test(ios): scope the submit-key guarantee to the same target; state table outcomes (#3334 review) The identity guard in replacementTextEntryResult returns TEXT_ENTRY_MISMATCH for a read resolving to a different element before the submit-key exception is reached, so the published guarantee's 'any non-matching value' is scoped to values observed on the same target in both surfaces that state it. The slow-read table's expected unconfirmed after-value becomes an explicit per- row literal instead of a check derived from the row's own input.
1 parent ca227e2 commit d71441e

13 files changed

Lines changed: 560 additions & 226 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 159 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ @interface AgentDeviceRunnerViewController : UIViewController
7171
// took focus from an input that has stopped answering the accessibility query that found it.
7272
@property(nonatomic, strong) UILabel *textEntryFocusWitness;
7373
@property(nonatomic, assign) NSUInteger textEntryWriteBacks;
74+
@property(nonatomic, assign) NSUInteger textEntryTotalEdits;
7475
@property(nonatomic, copy, nullable) NSString *textEntryRenderedValue;
7576
@property(nonatomic, assign) NSTimeInterval textEntryLastEditTime;
7677
@property(nonatomic, assign) NSTimeInterval textEntryBurstStartTime;
@@ -97,6 +98,122 @@ - (NSString *)accessibilityValue {
9798
return [NSString stringWithFormat:@"%lu of 6 digits", (unsigned long)self.text.length];
9899
}
99100
@end
101+
102+
// A field that reformats its own text on every edit into `## ### ## ##` digit groups, the way a
103+
// digit-grouping formatter in a Flutter or React Native input does. The accessibility value is
104+
// what the user sees — the formatted string — so it never equals the digits a `fill` sent (#2634).
105+
// The field owns the delegate it installs, keeping the formatter out of the view controller's
106+
// own delegate callbacks.
107+
@interface AgentDeviceDigitGroupingTextField : UITextField <UITextFieldDelegate>
108+
@end
109+
110+
@interface AgentDeviceCurrencyTextField : UITextField <UITextFieldDelegate>
111+
@property(nonatomic, strong) UILabel *entryStatus;
112+
@property(nonatomic, assign) NSUInteger insertedDigits;
113+
@end
114+
115+
@interface AgentDeviceSubmitClearingTextField : UITextField <UITextFieldDelegate>
116+
@property(nonatomic, strong) UILabel *entryStatus;
117+
@property(nonatomic, assign) NSUInteger submits;
118+
@end
119+
120+
static NSString *AgentDeviceFormatGroupedDigits(NSString *raw) {
121+
NSMutableString *digits = [NSMutableString string];
122+
NSCharacterSet *nonDigits = [[NSCharacterSet decimalDigitCharacterSet] invertedSet];
123+
for (NSString *chunk in [raw componentsSeparatedByCharactersInSet:nonDigits]) {
124+
[digits appendString:chunk];
125+
}
126+
NSMutableArray<NSNumber *> *groupWidths = [NSMutableArray arrayWithArray:@[@2, @3, @2, @2]];
127+
NSMutableString *formatted = [NSMutableString string];
128+
NSUInteger cursor = 0;
129+
for (NSNumber *width in groupWidths) {
130+
if (cursor >= digits.length) {
131+
break;
132+
}
133+
if (formatted.length > 0) {
134+
[formatted appendString:@" "];
135+
}
136+
NSUInteger end = MIN(cursor + width.unsignedIntegerValue, digits.length);
137+
[formatted appendString:[digits substringWithRange:NSMakeRange(cursor, end - cursor)]];
138+
cursor = end;
139+
}
140+
// Digits past the last group keep grouping by threes rather than vanishing from the field.
141+
while (cursor < digits.length) {
142+
NSUInteger end = MIN(cursor + 3, digits.length);
143+
[formatted appendString:[NSString stringWithFormat:@" %@", [digits substringWithRange:NSMakeRange(cursor, end - cursor)]]];
144+
cursor = end;
145+
}
146+
return formatted;
147+
}
148+
149+
static NSUInteger AgentDeviceDigitCount(NSString *text, NSUInteger limit) {
150+
NSUInteger count = 0;
151+
for (NSUInteger index = 0; index < limit && index < text.length; index++) {
152+
if ([NSCharacterSet.decimalDigitCharacterSet characterIsMember:[text characterAtIndex:index]]) {
153+
count += 1;
154+
}
155+
}
156+
return count;
157+
}
158+
159+
@implementation AgentDeviceDigitGroupingTextField
160+
- (BOOL)textField:(UITextField *)textField
161+
shouldChangeCharactersInRange:(NSRange)range
162+
replacementString:(NSString *)string {
163+
NSString *combined = [textField.text stringByReplacingCharactersInRange:range withString:string];
164+
NSString *formatted = AgentDeviceFormatGroupedDigits(combined);
165+
textField.text = formatted;
166+
// Assigning `text` drops the caret at the end, which a real grouping formatter does not do:
167+
// a mid-string edit must keep editing where the user put the caret. The digits before the
168+
// caret survive the reformat unchanged, so the caret belongs before the digit that follows
169+
// them — past the separators the grouping inserted behind them — and at the end when it
170+
// followed all of them. Typed digits count as before the caret; a delete does not move it.
171+
NSUInteger digitsBeforeCaret =
172+
AgentDeviceDigitCount(textField.text, range.location) + AgentDeviceDigitCount(string, string.length);
173+
NSUInteger caretOffset = formatted.length;
174+
NSUInteger seen = 0;
175+
for (NSUInteger index = 0; index < formatted.length; index++) {
176+
if (![NSCharacterSet.decimalDigitCharacterSet characterIsMember:[formatted characterAtIndex:index]]) {
177+
continue;
178+
}
179+
seen += 1;
180+
if (seen == digitsBeforeCaret + 1) {
181+
caretOffset = index;
182+
break;
183+
}
184+
}
185+
textField.selectedTextRange = [textField textRangeFromPosition:[textField positionFromPosition:textField.beginningOfDocument offset:caretOffset]
186+
toPosition:[textField positionFromPosition:textField.beginningOfDocument offset:caretOffset]];
187+
return NO;
188+
}
189+
@end
190+
191+
@implementation AgentDeviceCurrencyTextField
192+
- (BOOL)textField:(UITextField *)textField
193+
shouldChangeCharactersInRange:(NSRange)range
194+
replacementString:(NSString *)string {
195+
NSString *combined = [textField.text stringByReplacingCharactersInRange:range withString:string];
196+
NSMutableString *digits = [NSMutableString string];
197+
for (NSString *part in [combined componentsSeparatedByCharactersInSet:NSCharacterSet.decimalDigitCharacterSet.invertedSet]) {
198+
[digits appendString:part];
199+
}
200+
unsigned long long cents = digits.longLongValue;
201+
textField.text = [NSString stringWithFormat:@"$%llu.%02llu", cents / 100, cents % 100];
202+
self.insertedDigits += AgentDeviceDigitCount(string, string.length);
203+
self.entryStatus.text = [NSString stringWithFormat:@"%lu", (unsigned long)self.insertedDigits];
204+
textField.selectedTextRange = [textField textRangeFromPosition:textField.endOfDocument toPosition:textField.endOfDocument];
205+
return NO;
206+
}
207+
@end
208+
209+
@implementation AgentDeviceSubmitClearingTextField
210+
- (BOOL)textFieldShouldReturn:(UITextField *)textField {
211+
self.submits += 1;
212+
self.entryStatus.text = [NSString stringWithFormat:@"%lu", (unsigned long)self.submits];
213+
textField.text = @"";
214+
return YES;
215+
}
216+
@end
100217
#endif
101218

102219
@implementation AgentDeviceRunnerViewController
@@ -202,11 +319,12 @@ - (void)updateAlertActionStatus {
202319
- (void)updateTextEntryWriteBackStatus {
203320
NSTimeInterval burstSpan = self.textEntryLastEditTime - self.textEntryBurstStartTime;
204321
self.textEntryWriteBackStatus.text = [NSString
205-
stringWithFormat:@"write-backs=%lu burst-edits=%lu burst-ms=%lu min-gap-ms=%lu",
322+
stringWithFormat:@"write-backs=%lu burst-edits=%lu burst-ms=%lu min-gap-ms=%lu total-edits=%lu",
206323
(unsigned long)self.textEntryWriteBacks,
207324
(unsigned long)self.textEntryBurstEdits,
208325
(unsigned long)llround(burstSpan * 1000),
209-
(unsigned long)llround(self.textEntryBurstMinGap * 1000)];
326+
(unsigned long)llround(self.textEntryBurstMinGap * 1000),
327+
(unsigned long)self.textEntryTotalEdits];
210328
}
211329

212330
- (void)presentAlertFixtureReplacement:(BOOL)replacement {
@@ -292,6 +410,7 @@ - (void)agentDeviceTextEntryDidChange:(UITextField *)textField {
292410
// typed faster than the app renders loses the characters that arrived while a render was in
293411
// flight, and the field settles stable short of the request.
294412
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-app-owned-value"]) {
413+
self.textEntryTotalEdits += 1;
295414
NSTimeInterval now = NSProcessInfo.processInfo.systemUptime;
296415
NSTimeInterval gap = now - self.textEntryLastEditTime;
297416
BOOL overtookARender = self.textEntryRenderedValue != nil && gap < self.textEntryAcknowledgeWindowSeconds;
@@ -408,8 +527,28 @@ - (void)viewDidLoad {
408527
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-regression"]) {
409528
BOOL digitCountValue =
410529
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-digit-count-value"];
411-
UITextField *textField =
412-
digitCountValue ? [[AgentDeviceDigitCountTextField alloc] init] : [[UITextField alloc] init];
530+
BOOL digitGroupingValue =
531+
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-digit-grouping-value"];
532+
BOOL currencyValue =
533+
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-currency-value"];
534+
BOOL submitClears =
535+
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-submit-clears"];
536+
UITextField *textField;
537+
if (digitCountValue) {
538+
textField = [[AgentDeviceDigitCountTextField alloc] init];
539+
} else if (digitGroupingValue) {
540+
textField = [[AgentDeviceDigitGroupingTextField alloc] init];
541+
} else if (currencyValue) {
542+
textField = [[AgentDeviceCurrencyTextField alloc] init];
543+
textField.text = @"$0.00";
544+
} else if (submitClears) {
545+
textField = [[AgentDeviceSubmitClearingTextField alloc] init];
546+
} else {
547+
textField = [[UITextField alloc] init];
548+
}
549+
if (digitGroupingValue || currencyValue || submitClears) {
550+
textField.delegate = (id<UITextFieldDelegate>)textField;
551+
}
413552
if (![NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unnamed-input"]) {
414553
textField.accessibilityIdentifier = @"agent-device-hardware-keyboard-input";
415554
}
@@ -434,6 +573,22 @@ - (void)viewDidLoad {
434573
[textField.widthAnchor constraintEqualToConstant:240],
435574
[textField.heightAnchor constraintEqualToConstant:AgentDeviceTextEntryFieldHeight],
436575
]];
576+
if (currencyValue || submitClears) {
577+
UILabel *entryStatus = [[UILabel alloc] init];
578+
entryStatus.accessibilityIdentifier = @"agent-device-text-entry-events";
579+
entryStatus.text = @"0";
580+
entryStatus.translatesAutoresizingMaskIntoConstraints = NO;
581+
[self.view addSubview:entryStatus];
582+
[NSLayoutConstraint activateConstraints:@[
583+
[entryStatus.centerXAnchor constraintEqualToAnchor:self.view.centerXAnchor],
584+
[entryStatus.topAnchor constraintEqualToAnchor:textField.bottomAnchor constant:12],
585+
]];
586+
if (currencyValue) {
587+
((AgentDeviceCurrencyTextField *)textField).entryStatus = entryStatus;
588+
} else {
589+
((AgentDeviceSubmitClearingTextField *)textField).entryStatus = entryStatus;
590+
}
591+
}
437592
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unqueryable-on-focus"]) {
438593
self.textEntryFocusWitness = [[UILabel alloc] init];
439594
self.textEntryFocusWitness.accessibilityIdentifier = @"agent-device-text-entry-focus";

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedTextEntry.swift‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -522,11 +522,8 @@ extension RunnerTests {
522522
/// `awaitSynthesizedReplacementCommit` binds the real XCUI reads). The budget defaults to the
523523
/// shipped one, so only a test that is asking about time has to name it.
524524
///
525-
/// Only an exact match settles. A value with a hole in the middle ("ada@example" -> "aexample")
526-
/// is the corruption this wait exists to catch, and `.replacement` mode has no formatter or
527-
/// autocomplete carve-out: `isRepairableTextEntryMismatch` (RunnerTests+TextTyping.swift) treats
528-
/// every `.replacement` mismatch as repairable, because `fill` owns the whole field via
529-
/// select-all. A settled non-match is therefore always the wait's failure case.
525+
/// This route has no element-bound baseline for unconfirmed evidence. Only exact read-back
526+
/// confirms a commit; prefix progress extends the bounded wait without confirming the value.
530527
static func awaitSynthesizedReplacementCommitOutcome(
531528
expectedText: String,
532529
placeholder: String?,

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -63,10 +63,8 @@ extension RunnerTests {
6363
static let pollInterval: TimeInterval = 0.02
6464
static let warmupValueTimeout: TimeInterval = 0.4
6565
static let verificationStabilityWindow: TimeInterval = 0.2
66-
/// How long a replacement value may take to move off its baseline, and then how long an unechoed
67-
/// value may keep changing before it must hold for `verificationStabilityWindow`; the app renders
68-
/// the last characters' summary late.
69-
static let unconfirmedSettleCeiling: TimeInterval = 1.0
66+
/// Initial replacement observation budget; a late first read gets one stability window.
67+
static let replacementSettleCeiling: TimeInterval = 1.0
7068
/// How long the commit wait tolerates seeing NO further progress toward the expected value.
7169
/// Numerically the flat deadline this replaced, so a pipeline that delivers nothing is
7270
/// condemned at exactly the same instant it always was (see `SynthesizedCommitDeadline`).

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryConfirmation.swift‎

Lines changed: 63 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import XCTest
22

3-
// What a replacement's read-back can prove when the field's value does not echo the typed text.
3+
// Target-bound observations and the replacement read-back policy.
44
// The wire shape mirrors the cross-platform `FillUnconfirmedVerification` in
55
// packages/contracts/src/fill-evidence.ts.
66
extension RunnerTests {
@@ -39,41 +39,74 @@ extension RunnerTests {
3939
let target: TextEntryElementIdentity
4040
}
4141

42-
/// Whether `observed` can be a degraded copy of the residual `baseline` plus the request: it is in
43-
/// order inside baseline + request (dropped characters), or it contains the request in order and
44-
/// the baseline did not already (residual text), or it contains baseline + request in order. A
45-
/// value related to the entry in none of these ways, such as an OTP field announcing
46-
/// "6 of 6 digits", is the app's own representation, so retyping cannot make it match.
47-
static func textEntryValueEchoes(observed: String, expected: String, baseline: String) -> Bool {
48-
let request = textEntryRequestWithoutSubmitKeys(expected)
49-
let residualAndRequest = baseline + request
50-
return isOrderedSubsequence(observed, of: residualAndRequest)
51-
|| (isOrderedSubsequence(request, of: observed) && !isOrderedSubsequence(request, of: baseline))
52-
|| isOrderedSubsequence(residualAndRequest, of: observed)
42+
struct ReplacementTextEntryConfirmation {
43+
private let requested: String
44+
private let baseline: TextEntryObservation?
45+
private var deadline: Date
46+
private var latest: TextEntryObservation?
47+
private var stableSince: Date
48+
49+
init(requested: String, baseline: TextEntryObservation?, startedAt: Date) {
50+
self.requested = requested
51+
self.baseline = baseline
52+
deadline = startedAt.addingTimeInterval(TextEntryTiming.replacementSettleCeiling)
53+
stableSince = startedAt
54+
}
55+
56+
mutating func observe(_ observed: TextEntryObservation, at sampledAt: Date) -> TextEntryResult? {
57+
// A late first read still needs one stability window.
58+
if latest == nil {
59+
deadline = max(deadline, sampledAt.addingTimeInterval(TextEntryTiming.verificationStabilityWindow))
60+
}
61+
if latest.map({ observed.isSettled(with: $0) }) != true { stableSince = sampledAt }
62+
latest = observed
63+
let result = RunnerTests.replacementTextEntryResult(requested: requested, baseline: baseline, observed: observed)
64+
let settled = sampledAt.timeIntervalSince(stableSince) >= TextEntryTiming.verificationStabilityWindow
65+
let moved = baseline.map { !observed.isSettled(with: $0) } ?? true
66+
if settled && (moved || result.verified != false || sampledAt >= deadline) {
67+
return result
68+
}
69+
if sampledAt >= deadline {
70+
return TextEntryResult(
71+
verified: nil, repaired: false, expectedText: requested, observedText: observed.value,
72+
failure: .commitNotObserved
73+
)
74+
}
75+
return nil
76+
}
5377
}
5478

55-
/// Classifies a replacement whose read-back never matched. The entry is unconfirmed, not failed,
56-
/// only when the same element's value moved off its pre-entry baseline to one that does not echo
57-
/// the request; every other mismatch stays a failure.
58-
static func unconfirmedTextEntryEvidence(
79+
/// Literal read-back policy; a mismatch does not establish why the app changed the text.
80+
static func replacementTextEntryResult(
5981
requested: String,
6082
baseline: TextEntryObservation?,
6183
observed: TextEntryObservation?
62-
) -> TextEntryUnconfirmedEvidence? {
63-
guard !textEntryRequestWithoutSubmitKeys(requested).isEmpty,
64-
let baseline,
65-
let observed,
66-
baseline.identity.isSameElement(as: observed.identity),
67-
observed.value != baseline.value,
68-
!textEntryValueEchoes(observed: observed.value, expected: requested, baseline: baseline.value)
69-
else {
70-
return nil
84+
) -> TextEntryResult {
85+
func result(_ verified: Bool?) -> TextEntryResult {
86+
TextEntryResult(verified: verified, repaired: false, expectedText: requested, observedText: observed?.value)
87+
}
88+
guard let observed else { return result(nil) }
89+
if let baseline, !baseline.identity.isSameElement(as: observed.identity) {
90+
return result(false)
7191
}
72-
return TextEntryUnconfirmedEvidence(
73-
requested: requested,
74-
before: baseline.value,
75-
after: observed.value,
76-
target: observed.identity
92+
let request = observed.identity.elementType == elementTypeNamesByRawValue[XCUIElement.ElementType.textView.rawValue]
93+
? requested : textEntryRequestWithoutSubmitKeys(requested)
94+
if observed.value == requested || observed.value == request { return result(true) }
95+
// Any non-matching single-line submit result remains unverified, even a partial value.
96+
if request != requested { return result(nil) }
97+
guard !request.isEmpty, let baseline else { return result(false) }
98+
if isOrderedSubsequence(observed.value, of: baseline.value + request) { return result(false) }
99+
return TextEntryResult(
100+
verified: nil,
101+
repaired: false,
102+
expectedText: requested,
103+
observedText: observed.value,
104+
unconfirmed: TextEntryUnconfirmedEvidence(
105+
requested: requested,
106+
before: baseline.value,
107+
after: observed.value,
108+
target: observed.identity
109+
)
77110
)
78111
}
79112

0 commit comments

Comments
 (0)