Skip to content

Commit 80bf831

Browse files
committed
fix(ios-runner): bound synthesized delivery by the main-thread watchdog and pin the pace the app sees
A 358-character fill through the daemon was abandoned by the runner's 30 s main-thread watchdog with the burst still typing. The delivery ceiling was a free-standing 30 s sized against the 45 s transport budget, while the whole command runs under mainThreadExecutionTimeout. The ceiling is now derived from that watchdog minus a focus allowance and the commit wait's ceiling (18 s), so the admitted text fits with the longest commit wait: 214 characters undelayed, 57 at --delay-ms 80. The app-owned-value lane test accepted either outcome and stayed green at 60 characters per second. It now asserts the spacing the fixture app receives: a burst's characters average at least one acknowledge window apart. That is red at 60 (11 edits in about 110 ms) and green at 12 (about 780 ms). A strict "the app keeps up" assertion is not possible: at a 40 ms window, 12 characters per second still lost one character in 13 of 20 bursts, because XCTest spaces the characters unevenly. The comments that claimed survival are narrowed to match. The call overhead and the --delay-ms 80 recovery delay move into TextEntryTiming beside the other text-entry timings, and both hints read the recovery delay from there.
1 parent a27335e commit 80bf831

12 files changed

Lines changed: 189 additions & 169 deletions

‎apple/runner/AgentDeviceRunner/AgentDeviceRunner/AgentDeviceRunnerApp.m‎

Lines changed: 36 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ @interface AgentDeviceRunnerViewController : UIViewController
7070
@property(nonatomic, assign) NSUInteger textEntryWriteBacks;
7171
@property(nonatomic, copy, nullable) NSString *textEntryRenderedValue;
7272
@property(nonatomic, assign) NSTimeInterval textEntryLastEditTime;
73+
@property(nonatomic, assign) NSTimeInterval textEntryBurstStartTime;
74+
@property(nonatomic, assign) NSUInteger textEntryBurstEdits;
75+
@property(nonatomic, assign) NSTimeInterval textEntryBurstMinGap;
7376
@property(nonatomic, assign) NSTimeInterval textEntryAcknowledgeWindowSeconds;
7477
@property(nonatomic, assign) BOOL alertFixtureStarted;
7578
@property(nonatomic, strong) NSTimer *alertActivationBusyBackstop;
@@ -182,9 +185,14 @@ - (void)updateAlertActionStatus {
182185
}
183186

184187
- (void)updateTextEntryWriteBackStatus {
185-
self.textEntryWriteBackStatus.text = [NSString stringWithFormat:@"Edits: %lu; write-backs: %lu",
186-
(unsigned long)self.textEntryRenderedEdits,
187-
(unsigned long)self.textEntryWriteBacks];
188+
NSTimeInterval burstSpan = self.textEntryLastEditTime - self.textEntryBurstStartTime;
189+
self.textEntryWriteBackStatus.text = [NSString
190+
stringWithFormat:@"edits=%lu write-backs=%lu burst-edits=%lu burst-ms=%lu min-gap-ms=%lu",
191+
(unsigned long)self.textEntryRenderedEdits,
192+
(unsigned long)self.textEntryWriteBacks,
193+
(unsigned long)self.textEntryBurstEdits,
194+
(unsigned long)llround(burstSpan * 1000),
195+
(unsigned long)llround(self.textEntryBurstMinGap * 1000)];
188196
}
189197

190198
- (void)presentAlertFixtureReplacement:(BOOL)replacement {
@@ -244,32 +252,39 @@ - (void)viewDidAppear:(BOOL)animated {
244252
}
245253
#endif
246254

247-
// How fast an app that owns this field's value can acknowledge edits: one render per window. An
248-
// edit that arrives inside that window overtook the render still in flight, so the value that
249-
// render commits predates it and writing it erases the characters that got ahead of the app. The
250-
// app then reads its own erasure back into its model, which is why the field stays wrong instead of
251-
// healing when the burst finishes. The window is decided at the edit rather than scheduled, so a
252-
// loaded host, which stretches the gaps between characters, can only make this app keep up better.
253-
static const NSTimeInterval AgentDeviceTextEntryDefaultAcknowledgeWindowSeconds = 0.04;
254-
255-
static NSTimeInterval AgentDeviceTextEntryAcknowledgeWindow(id argument) {
255+
// How fast an app that owns this field's value can acknowledge edits: one render per window, passed
256+
// by the test as `--agent-device-text-entry-acknowledge-window <seconds>`. An edit that arrives
257+
// inside that window overtook the render still in flight, so the value that render commits predates
258+
// it and writing it erases the characters that got ahead of the app. The app then reads its own
259+
// erasure back into its model, which is why the field stays wrong instead of healing when the burst
260+
// finishes. The window is decided at the edit rather than scheduled, so a loaded host, which
261+
// stretches the gaps between characters, can only make this app keep up better.
262+
static NSTimeInterval AgentDeviceTextEntryAcknowledgeWindow(void) {
256263
NSArray<NSString *> *arguments = NSProcessInfo.processInfo.arguments;
257-
NSUInteger index = [arguments indexOfObject:argument];
258-
if (index == NSNotFound || index + 1 >= arguments.count) {
259-
return AgentDeviceTextEntryDefaultAcknowledgeWindowSeconds;
260-
}
261-
NSTimeInterval seconds = [arguments[index + 1] doubleValue];
262-
return seconds > 0 ? seconds : AgentDeviceTextEntryDefaultAcknowledgeWindowSeconds;
264+
NSUInteger index = [arguments indexOfObject:@"--agent-device-text-entry-acknowledge-window"];
265+
return index == NSNotFound || index + 1 >= arguments.count ? 0 : [arguments[index + 1] doubleValue];
263266
}
264267

268+
// Edits further apart than this belong to different bursts: one runner command's characters arrive
269+
// well inside it, and two commands are separated by at least a commit-wait poll and a status read.
270+
static const NSTimeInterval AgentDeviceTextEntryBurstBreakSeconds = 1.0;
271+
265272
- (void)agentDeviceTextEntryDidChange:(UITextField *)textField {
266273
// A field whose app owns its value, the way a controlled React Native `TextInput` does. A burst
267274
// typed faster than the app renders loses the characters that arrived while a render was in
268275
// flight, and the field settles stable short of the request.
269276
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-app-owned-value"]) {
270277
NSTimeInterval now = NSProcessInfo.processInfo.systemUptime;
271-
BOOL overtookARender = self.textEntryRenderedValue != nil &&
272-
(now - self.textEntryLastEditTime) < self.textEntryAcknowledgeWindowSeconds;
278+
NSTimeInterval gap = now - self.textEntryLastEditTime;
279+
BOOL overtookARender = self.textEntryRenderedValue != nil && gap < self.textEntryAcknowledgeWindowSeconds;
280+
if (self.textEntryBurstEdits == 0 || gap > AgentDeviceTextEntryBurstBreakSeconds) {
281+
self.textEntryBurstStartTime = now;
282+
self.textEntryBurstEdits = 0;
283+
self.textEntryBurstMinGap = 0;
284+
} else if (self.textEntryBurstEdits == 1 || gap < self.textEntryBurstMinGap) {
285+
self.textEntryBurstMinGap = gap;
286+
}
287+
self.textEntryBurstEdits += 1;
273288
self.textEntryLastEditTime = now;
274289
if (overtookARender) {
275290
if (![textField.text isEqualToString:self.textEntryRenderedValue]) {
@@ -343,8 +358,7 @@ - (void)viewDidLoad {
343358
[textField.heightAnchor constraintEqualToConstant:44],
344359
]];
345360
if ([NSProcessInfo.processInfo.arguments containsObject:@"--agent-device-text-entry-app-owned-value"]) {
346-
self.textEntryAcknowledgeWindowSeconds =
347-
AgentDeviceTextEntryAcknowledgeWindow(@"--agent-device-text-entry-acknowledge-window");
361+
self.textEntryAcknowledgeWindowSeconds = AgentDeviceTextEntryAcknowledgeWindow();
348362
// Reports how many edits this app rendered and how many writes it had to make because a
349363
// character overtook one, so a lane test can tell a burst the app kept up with from an inert
350364
// fixture. Counts only: no field content crosses into the test.

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerSynthesizedTextEntry.m‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,13 @@
1010
// its field's value and re-applies it after the edit (a controlled React Native `TextInput`, an
1111
// async validator) can acknowledge: such a write lands between two characters of the burst and
1212
// erases what was typed while it was in flight, leaving a value that is stable short of the
13-
// request. 12 characters/second spaces them ~83 ms apart, so an app whose write-back lands inside
14-
// one character interval no longer has anything to erase. It is not immunity: a render still in
15-
// flight 150 ms after an edit corrupted an 11-character burst at this pace too. The
16-
// `--agent-device-text-entry-app-owned-value` fixture and the pace policy test pin this bound; the
17-
// delivery ceiling in TextEntryTiming bounds what the pace costs a long text.
13+
// request. 12 characters/second spaces them ~83 ms apart on average, which reduces that loss but
14+
// does not remove it: XCTest does not space the characters evenly, and two of them can reach the
15+
// app a few milliseconds apart. Against a fixture app that acknowledges each edit within 40 ms, 60
16+
// characters/second left 1 of 11 characters in 20 of 20 bursts, and this pace left 10 or 11. The
17+
// command refuses a field left short; back-pressure from the field (#2906) is what would prevent
18+
// it. The app-owned-value lane test pins the average spacing the app sees, and the delivery budget
19+
// in TextEntryTiming bounds what the pace costs a long text.
1820
static const NSUInteger RunnerTextEntryTypingSpeedCharactersPerSecond = 12;
1921

2022
typedef id (*RunnerTextMsgSendInit)(id, SEL, NSString *);

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ extension RunnerTests {
144144
}
145145
return try runMainThreadWork(
146146
"command_execution",
147-
timeout: mainThreadExecutionTimeout,
147+
timeout: Self.mainThreadExecutionTimeout,
148148
timeoutError: mainThreadExecutionTimeoutError
149149
) {
150150
try self.executeOnMainSafely(command: command, routeToSpringboard: routeToSpringboard)
@@ -247,7 +247,7 @@ extension RunnerTests {
247247
while true {
248248
let failureCountBefore = try runMainThreadWork(
249249
"recorded_failure_count",
250-
timeout: mainThreadExecutionTimeout,
250+
timeout: Self.mainThreadExecutionTimeout,
251251
timeoutError: mainThreadExecutionTimeoutError
252252
) {
253253
self.currentXCTestFailureCount()
@@ -264,7 +264,7 @@ extension RunnerTests {
264264
}
265265
let recordedFailureResponse = try runMainThreadWork(
266266
"recorded_failure_count",
267-
timeout: mainThreadExecutionTimeout,
267+
timeout: Self.mainThreadExecutionTimeout,
268268
timeoutError: mainThreadExecutionTimeoutError
269269
) {
270270
self.didRecordXCTestFailure(since: failureCountBefore)
@@ -274,7 +274,7 @@ extension RunnerTests {
274274
if let recordedFailureResponse {
275275
try runMainThreadWork(
276276
"target_invalidation",
277-
timeout: mainThreadExecutionTimeout,
277+
timeout: Self.mainThreadExecutionTimeout,
278278
timeoutError: mainThreadExecutionTimeoutError
279279
) {
280280
self.invalidateCachedTarget(reason: "xctest_recorded_failure")
@@ -289,7 +289,7 @@ extension RunnerTests {
289289
hasRetried = true
290290
try runMainThreadWork(
291291
"target_invalidation",
292-
timeout: mainThreadExecutionTimeout,
292+
timeout: Self.mainThreadExecutionTimeout,
293293
timeoutError: mainThreadExecutionTimeoutError
294294
) {
295295
self.invalidateCachedTarget(reason: "response_unavailable")

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ extension RunnerTests {
1111
private func executeSnapshotDispatchedOnce(command: Command) throws -> Response {
1212
let preparation: SnapshotCommandPreparation = try runMainThreadWork(
1313
"command_preparation",
14-
timeout: mainThreadExecutionTimeout,
14+
timeout: Self.mainThreadExecutionTimeout,
1515
timeoutError: mainThreadExecutionTimeoutError
1616
) { () -> SnapshotCommandPreparation in
1717
switch try self.prepareActiveCommandContextSafely(command: command, routeToSpringboard: false) {

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedTextEntry.swift‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -109,11 +109,6 @@ extension RunnerTests {
109109
1.0 / Double(RunnerSynthesizedTextEntry.typingSpeedCharactersPerSecond())
110110
}
111111

112-
/// Seconds one synthesize call costs beyond typing its characters. A one-character call at the
113-
/// 83 ms pace took 222 ms on average on an iPhone 17 Pro simulator (212-617 ms over 235 calls),
114-
/// which a spaced plan pays once per character.
115-
static let synthesizeCallOverhead: TimeInterval = 0.15
116-
117112
/// Seconds the plan spends posting, charged per `synthesizedReplacementSteps` step: each
118113
/// synthesize call types its characters at the pace and pays its overhead, and a spaced plan
119114
/// sleeps `delaySeconds` between two calls.
@@ -122,7 +117,7 @@ extension RunnerTests {
122117
? textLength
123118
: 1
124119
return Double(textLength) * characterInterval
125-
+ Double(calls) * synthesizeCallOverhead
120+
+ Double(calls) * TextEntryTiming.synthesizeCallOverhead
126121
+ Double(calls - 1) * delaySeconds
127122
}
128123

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift‎

Lines changed: 28 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,13 @@ extension RunnerTests {
3030
case .synthesisUnavailable:
3131
return "Show the software keyboard, then retry type."
3232
case .commitNotObserved:
33-
return "The field may hold none, part, or all of the text. Run snapshot -i and inspect the field: if it already matches, continue; otherwise retry fill with the full text quoted and --delay-ms 80. Do not use type, which appends to whatever committed."
33+
return "The field may hold none, part, or all of the text. Run snapshot -i and inspect the field: if it already matches, continue; otherwise retry fill with the full text quoted and --delay-ms \(TextEntryTiming.recoveryDelayMilliseconds). Do not use type, which appends to whatever committed."
3434
case .synthesisBudgetExceeded:
35-
return "Fill at most \(SynthesizedDeliveryBudget.maxTextLength(delaySeconds: 0)) characters at a time without --delay-ms and append the rest with separate type commands, keeping each command inside that budget. --delay-ms lowers the budget, because every character then pays its own synthesize call and the delay: \(SynthesizedDeliveryBudget.maxTextLength(delaySeconds: 0.08)) characters at --delay-ms 80. A longer timeout does not help: this route is chosen when the accessibility channel is already degraded, and the pace is what makes the text long."
35+
let recoveryDelay = TextEntryTiming.recoveryDelayMilliseconds
36+
let recoveryBudget = SynthesizedDeliveryBudget.maxTextLength(
37+
delaySeconds: Double(recoveryDelay) / 1000
38+
)
39+
return "Fill at most \(SynthesizedDeliveryBudget.maxTextLength(delaySeconds: 0)) characters at a time without --delay-ms and append the rest with separate type commands, keeping each command inside that budget. --delay-ms lowers the budget, because every character then pays its own synthesize call and the delay: \(recoveryBudget) characters at --delay-ms \(recoveryDelay). A longer timeout does not help: this route is chosen when the accessibility channel is already degraded, and the pace is what makes the text long."
3640
}
3741
}
3842
}
@@ -55,22 +59,30 @@ extension RunnerTests {
5559
/// condemned at exactly the same instant it always was (see `SynthesizedCommitDeadline`).
5660
static let synthesizedCommitStallTimeout: TimeInterval = 3.0
5761
/// The commit wait's absolute bound, however long characters keep arriving. Synthesized
58-
/// delivery happens before this wait starts and is bounded by `synthesizedDeliveryCeiling`, so
59-
/// the two together stay inside the daemon's per-command budget
60-
/// (`RUNNER_COMMAND_TIMEOUT_MS`, 45s), which also has to cover focus, clear and verification.
62+
/// delivery happens before this wait starts and is bounded by `synthesizedDeliveryCeiling`.
6163
static let synthesizedCommitCeiling: TimeInterval = 10.0
62-
/// How long a synthesized burst may spend posting its characters. The private synthesize call
63-
/// delivers as it returns, so this is the slice of the 45s command budget the burst itself may
64-
/// take, with the commit ceiling, focus, clear and verification subtracted and margin left for
65-
/// the round trip each character costs. Text that does not fit is refused before the first
66-
/// character is posted: a transport timeout would end the command with the runner still typing,
67-
/// and the next command would find it busy.
68-
static let synthesizedDeliveryCeiling: TimeInterval = 30.0
69-
/// The edit-acknowledge budget the synthesized pace is sized for: an app that renders each edit
70-
/// within this window has nothing to erase when a burst replaces its field, and one that needs
71-
/// longer loses the characters that arrive while a render is in flight.
72-
/// `testSynthesizedPaceLeavesRoomForAnAppToAcknowledgeEachEdit` pins the pace against it.
64+
/// What a synthesized replacement spends before its first character: focusing the field took
65+
/// 374–500 ms through the daemon on an iPhone 17 Pro simulator.
66+
static let synthesizedReplacementFocusAllowance: TimeInterval = 2.0
67+
/// How long a synthesized burst may spend posting its characters: what the command's
68+
/// main-thread watchdog leaves after focus and the longest commit wait. The private synthesize
69+
/// call delivers as it returns, so text that does not fit is refused before the first character
70+
/// is posted; otherwise the watchdog abandons the command with the runner still typing.
71+
static let synthesizedDeliveryCeiling: TimeInterval = RunnerTests.mainThreadExecutionTimeout
72+
- synthesizedReplacementFocusAllowance
73+
- synthesizedCommitCeiling
74+
/// The edit-acknowledge window the synthesized pace is sized for: on average, a burst's
75+
/// characters reach the app at least this far apart. XCTest spaces them unevenly, so an app
76+
/// with this window can still lose a character that arrives early; the command then refuses the
77+
/// short value (#2906 tracks preventing it). The pace policy test and the app-owned-value lane
78+
/// test pin the pace against it.
7379
static let synthesizedAcknowledgeWindowSeconds: TimeInterval = 0.04
80+
/// What one private synthesize call costs beyond typing its characters, which a `--delay-ms`
81+
/// plan pays once per character. One-character calls at the shipped pace took 222 ms on average
82+
/// on an iPhone 17 Pro simulator (212–617 ms over 235 calls), 83 ms of it the character.
83+
static let synthesizeCallOverhead: TimeInterval = 0.15
84+
/// The spacing the `TEXT_INPUT_COMMIT_NOT_OBSERVED` recovery tells the caller to retry with.
85+
static let recoveryDelayMilliseconds = 80
7486
static let synthesizedCommitPollInterval: TimeInterval = 0.2
7587
}
7688

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextTyping.swift‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -163,9 +163,9 @@ extension RunnerTests {
163163
} else if activeTarget.prefersFocusedElement && isKeyboardVisible(app: app) {
164164
#if os(iOS)
165165
// Text the command budget cannot carry at the synthesized pace goes through the verified
166-
// application-wide typing instead. The synthesizer types at the pace an app that owns its
167-
// field can acknowledge, and a burst that long outlasts the command while the runner is
168-
// still posting it. `app.typeText` is this branch's existing fallback and its value is
166+
// application-wide typing instead. The synthesizer's pace is slowed for fields whose app
167+
// owns the value, and a burst that long outlasts the command while the runner is still
168+
// posting it. `app.typeText` is this branch's existing fallback and its value is
169169
// verified afterwards, so the length costs the pace, not the check.
170170
if SynthesizedDeliveryBudget.exceeds(textLength: value.count, delaySeconds: 0) {
171171
textEntryRoute = "xctest-application-fallback"

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ final class RunnerTests: XCTestCase {
6868
// interactions clear it before it can become stale.
6969
var textEntryTapWitness: TextEntryTapWitness?
7070
let maxRequestBytes = 2 * 1024 * 1024
71-
let mainThreadExecutionTimeout: TimeInterval = 30
71+
static let mainThreadExecutionTimeout: TimeInterval = 30
7272
let appExistenceTimeout: TimeInterval = 30
7373
let retryCooldown: TimeInterval = 0.2
7474
let postSnapshotInteractionDelay: TimeInterval = 0.2

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+RecordingTests.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -448,7 +448,7 @@ extension RunnerTests {
448448
for _ in 0..<2 {
449449
try self.runMainThreadWork(
450450
"command_execution",
451-
timeout: self.mainThreadExecutionTimeout,
451+
timeout: Self.mainThreadExecutionTimeout,
452452
timeoutError: self.mainThreadExecutionTimeoutError
453453
) {
454454
Thread.sleep(forTimeInterval: self.recordingFrameCaptureTimeout + 0.3)

0 commit comments

Comments
 (0)