Skip to content

Commit 3244fec

Browse files
thymikeeclaude
andcommitted
Merge origin/main into perf/ios-fill-runner-cost
Re-express the input-first focus confirmation in #3882's read model. The tapped input's read is a ResolvedTextEntryInput; when its snapshot reports keyboard focus, that same read gives the refresh point and seeds the TextEntryReader as the focus step's post-focus read. keyboardFocusedInputFrame and the frame-only refresh path are gone. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2 parents fcc403f + 4d5c384 commit 3244fec

41 files changed

Lines changed: 1403 additions & 392 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ The versioned CLI help owns command semantics. To drive an app, start with the s
4848
- In a new worktree, run `pnpm install --frozen-lockfile && pnpm build` before you test.
4949
- Do not change other worktrees, devices, sessions, dirty files, or experiments. Do not use `git stash`; all worktrees share it.
5050
- Do not copy `.env` into a worktree. For host credentials, source the main checkout's file by its absolute path.
51-
- Give each parallel worker different files and a different device. `pnpm check:affected --run` and `pnpm check` run one at a time automatically.
51+
- Give each parallel worker different files and a different device. `pnpm check:affected --run`, `pnpm check`, and heavy `pnpm gate` ids share one lock.
5252
- The layering scan reads only tracked files. Stage new modules before you trust its result.
5353
- When you rename a file, move its Fallow baseline entry. Do not regenerate unrelated baselines.
5454

‎CONTRIBUTING.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,11 @@ Append `:clean` to any platform build when DerivedData may be stale, for example
5555
`pnpm build:xcuitest:macos:clean`. `pnpm build:xcuitest` remains the shared iOS-and-macOS gate for
5656
changes that affect both runners; it is not an all-platform build.
5757

58+
`pnpm clean:xcuitest[:ios|:macos|:tvos|:visionos]` cleans the runner derived-data directory and prints
59+
it. When `AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH` is set, the scripts clean only that directory, for
60+
every platform, and keep entries that are not build output. Otherwise they clean the default
61+
directory under `~/.agent-device/apple-runner/derived`.
62+
5863
Android and macOS helper builds remain separate because they require their native toolchains:
5964

6065
```bash

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 87 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ @interface AgentDeviceRunnerViewController : UIViewController
110110
@property(nonatomic, assign) NSTimeInterval textEntryBurstMinGap;
111111
@property(nonatomic, assign) NSTimeInterval textEntryAcknowledgeWindowSeconds;
112112
@property(nonatomic, strong) NSLayoutConstraint *textEntryFieldTop;
113+
@property(nonatomic, weak) UITextField *textEntryField;
113114
@property(nonatomic, assign) BOOL alertFixtureStarted;
114115
@property(nonatomic, strong) NSTimer *alertActivationBusyBackstop;
115116
@property(nonatomic, strong) NSTimer *alertBannerRepost;
@@ -130,6 +131,39 @@ - (NSString *)accessibilityValue {
130131
}
131132
@end
132133

134+
// XCTest reads every element's accessibility value each time it snapshots the app, which it does
135+
// for every element query and every attribute read from a live element. Counting this field's
136+
// value reads therefore counts the accessibility queries a command makes. The value is masked, so a
137+
// fill's read-back ends at its first read instead of polling through a stability window whose
138+
// poll count depends on how fast the host answers.
139+
static NSUInteger AgentDeviceTextEntryValueReads = 0;
140+
141+
@interface AgentDeviceValueReadCountingTextField : AgentDeviceMaskedTextField
142+
@end
143+
144+
@implementation AgentDeviceValueReadCountingTextField
145+
- (NSString *)accessibilityValue {
146+
AgentDeviceTextEntryValueReads += 1;
147+
return [super accessibilityValue];
148+
}
149+
@end
150+
151+
// Reports the read count as its accessibility label and the counted field's text as its value, so
152+
// reading either changes no view state and the masked text stays checkable.
153+
@interface AgentDeviceValueReadCountLabel : UILabel
154+
@property(nonatomic, weak) UITextField *countedField;
155+
@end
156+
157+
@implementation AgentDeviceValueReadCountLabel
158+
- (NSString *)accessibilityLabel {
159+
return [NSString stringWithFormat:@"%lu", (unsigned long)AgentDeviceTextEntryValueReads];
160+
}
161+
162+
- (NSString *)accessibilityValue {
163+
return self.countedField.text;
164+
}
165+
@end
166+
133167
// A field that reformats its own text on every edit into `## ### ## ##` digit groups, the way a
134168
// digit-grouping formatter in a Flutter or React Native input does. The accessibility value is
135169
// what the user sees — the formatted string — so it never equals the digits a `fill` sent (#2634).
@@ -513,20 +547,33 @@ - (void)agentDeviceTextEntryDidChange:(UITextField *)textField {
513547
}
514548
if ((codeComplete && [arguments containsObject:@"--agent-device-text-entry-auto-submit"]) ||
515549
(textField.text.length > 0 && [arguments containsObject:@"--agent-device-text-entry-replace-after-input"])) {
516-
UITextField *nextScreenField = [[UITextField alloc] initWithFrame:textField.frame];
517-
if (![arguments containsObject:@"--agent-device-text-entry-unnamed-input"]) {
518-
nextScreenField.accessibilityIdentifier = @"agent-device-auto-submit-next-screen-input";
519-
}
520-
nextScreenField.borderStyle = UITextBorderStyleRoundedRect;
521-
[textField.superview addSubview:nextScreenField];
522-
[textField removeFromSuperview];
523-
[nextScreenField becomeFirstResponder];
550+
[self replaceTextEntryFieldWithNextScreenInput:textField];
524551
} else if (codeComplete &&
525552
[arguments containsObject:@"--agent-device-text-entry-auto-submit-without-successor"]) {
526553
[textField removeFromSuperview];
527554
}
528555
}
529556

557+
- (void)replaceTextEntryFieldWithNextScreenInput:(UITextField *)textField {
558+
UITextField *nextScreenField = [[UITextField alloc] initWithFrame:textField.frame];
559+
if (![NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unnamed-input"]) {
560+
nextScreenField.accessibilityIdentifier = @"agent-device-auto-submit-next-screen-input";
561+
}
562+
nextScreenField.borderStyle = UITextBorderStyleRoundedRect;
563+
[textField.superview addSubview:nextScreenField];
564+
[textField removeFromSuperview];
565+
[nextScreenField becomeFirstResponder];
566+
}
567+
568+
// The same navigation as `replace-after-input`, landing when a test presses the button instead of on
569+
// an edit, so the test decides which runner step it lands between.
570+
- (void)agentDeviceTextEntryReplaceDidPress:(UIButton *)sender {
571+
(void)sender;
572+
if (self.textEntryField.superview != nil) {
573+
[self replaceTextEntryFieldWithNextScreenInput:self.textEntryField];
574+
}
575+
}
576+
530577
// Stops answering the element-type query that found it, the moment it takes focus, under
531578
// `--agent-device-text-entry-unqueryable-on-focus`. #3060's Flutter password field reaches this shape
532579
// because its two accessibility channels disagree once focused — `TextField` through the legacy
@@ -690,6 +737,8 @@ - (void)viewDidLoad {
690737
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-submit-clears"];
691738
BOOL maskedValue =
692739
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-masked-value"];
740+
BOOL countsValueReads =
741+
[NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-counts-value-reads"];
693742
UITextField *textField;
694743
if (digitCountValue) {
695744
textField = [[AgentDeviceDigitCountTextField alloc] init];
@@ -702,6 +751,8 @@ - (void)viewDidLoad {
702751
textField = [[AgentDeviceSubmitClearingTextField alloc] init];
703752
} else if (maskedValue) {
704753
textField = [[AgentDeviceMaskedTextField alloc] init];
754+
} else if (countsValueReads) {
755+
textField = [[AgentDeviceValueReadCountingTextField alloc] init];
705756
} else {
706757
textField = [[UITextField alloc] init];
707758
}
@@ -724,6 +775,7 @@ - (void)viewDidLoad {
724775
forControlEvents:UIControlEventEditingChanged];
725776
textField.translatesAutoresizingMaskIntoConstraints = NO;
726777
[self.view addSubview:textField];
778+
self.textEntryField = textField;
727779
self.textEntryFieldTop = [textField.topAnchor constraintEqualToAnchor:label.bottomAnchor
728780
constant:AgentDeviceTextEntryFieldTopInset];
729781
[NSLayoutConstraint activateConstraints:@[
@@ -732,6 +784,18 @@ - (void)viewDidLoad {
732784
[textField.widthAnchor constraintEqualToConstant:240],
733785
[textField.heightAnchor constraintEqualToConstant:AgentDeviceTextEntryFieldHeight],
734786
]];
787+
if (countsValueReads) {
788+
AgentDeviceValueReadCountLabel *valueReads = [[AgentDeviceValueReadCountLabel alloc] init];
789+
valueReads.countedField = textField;
790+
valueReads.accessibilityIdentifier = @"agent-device-text-entry-value-reads";
791+
valueReads.text = @"value reads";
792+
valueReads.translatesAutoresizingMaskIntoConstraints = NO;
793+
[self.view addSubview:valueReads];
794+
[NSLayoutConstraint activateConstraints:@[
795+
[valueReads.centerXAnchor constraintEqualToAnchor:self.view.centerXAnchor],
796+
[valueReads.topAnchor constraintEqualToAnchor:textField.bottomAnchor constant:12],
797+
]];
798+
}
735799
if (currencyValue || submitClears) {
736800
UILabel *entryStatus = [[UILabel alloc] init];
737801
entryStatus.accessibilityIdentifier = @"agent-device-text-entry-events";
@@ -774,6 +838,21 @@ - (void)viewDidLoad {
774838
]];
775839
[self updateTapWitnessStatus];
776840
}
841+
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-replace-on-request"]) {
842+
// Above the title, so the software keyboard never covers it.
843+
UIButton *replace = [UIButton buttonWithType:UIButtonTypeSystem];
844+
replace.accessibilityIdentifier = @"agent-device-text-entry-replace";
845+
[replace setTitle:@"Replace input" forState:UIControlStateNormal];
846+
replace.translatesAutoresizingMaskIntoConstraints = NO;
847+
[replace addTarget:self
848+
action:@selector(agentDeviceTextEntryReplaceDidPress:)
849+
forControlEvents:UIControlEventTouchUpInside];
850+
[self.view addSubview:replace];
851+
[NSLayoutConstraint activateConstraints:@[
852+
[replace.centerXAnchor constraintEqualToAnchor:self.view.centerXAnchor],
853+
[replace.bottomAnchor constraintEqualToAnchor:label.topAnchor constant:-24],
854+
]];
855+
}
777856
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-unqueryable-on-focus"]) {
778857
self.textEntryFocusWitness = [[UILabel alloc] init];
779858
self.textEntryFocusWitness.accessibilityIdentifier = @"agent-device-text-entry-focus";

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ extension RunnerTests {
335335

336336
/// Shared ordering for point-hit candidates: smallest area wins, then top-to-bottom,
337337
/// left-to-right, then stable element-type order for ties.
338-
func smallestElementFirst(_ left: XCUIElement, _ right: XCUIElement) -> Bool {
338+
func smallestElementFirst(_ left: any XCUIElementAttributes, _ right: any XCUIElementAttributes) -> Bool {
339339
let leftArea = max(1, left.frame.width * left.frame.height)
340340
let rightArea = max(1, right.frame.width * right.frame.height)
341341
if leftArea != rightArea {
@@ -352,7 +352,7 @@ extension RunnerTests {
352352

353353
func readTextAt(app: XCUIApplication, x: Double, y: Double) -> String? {
354354
let point = CGPoint(x: x, y: y)
355-
let textInputCandidates = textInputCandidatesAt(app: app, point: point)
355+
let textInputCandidates = textInputCandidatesAt(app: app, point: point).map(\.element)
356356
for element in textInputCandidates where prefersExpandedTextRead(element) {
357357
if let text = readableText(for: element) {
358358
return text
@@ -363,7 +363,7 @@ extension RunnerTests {
363363
.filter { element in
364364
element.exists && !element.frame.isEmpty && element.frame.contains(point)
365365
}
366-
.sorted(by: smallestElementFirst)
366+
.sorted { smallestElementFirst($0, $1) }
367367

368368
for element in candidates where prefersExpandedTextRead(element) {
369369
if let text = readableText(for: element) {

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedCommitDeadline.swift‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -138,15 +138,15 @@ extension RunnerTests {
138138
observe: () -> TextValueRead,
139139
waitForNextObservation: () -> Void
140140
) {
141-
let field = resolveTextEntryElement(app: app, target: target)
141+
let field = resolveTextEntryInput(app: app, target: target)?.attributes
142142
let placeholder = field?.placeholderValue
143143
let waitStartedAt = Date()
144144
return (
145145
placeholder: placeholder,
146146
fieldIsUnreadable: textValueRead(for: field) == .unreadable,
147147
observe: {
148148
let observed = self.textValueRead(
149-
for: self.resolveTextEntryElement(app: app, target: target),
149+
for: self.resolveTextEntryInput(app: app, target: target)?.attributes,
150150
treatingPlaceholderAsEmpty: true
151151
)
152152
var observedText: String?

0 commit comments

Comments
 (0)