Skip to content

Commit 2df8924

Browse files
committed
fix(apple-runner): serve a macOS read of a background app without raising it
On a desktop, the XCTest foreground repair is not a no-op for the user: a read that names a session app sitting behind other windows took their frontmost app away for a command that only asked to look (#3254). The activation axis already routes by `launchPolicy`, so the fix goes on the existing `.existingApp` arm of `prepareActiveCommandContext`: on macOS, a read whose app answers `.runningBackground` is served in place through `resolveAppWithoutActivation`, and no activation fact is booked for it. Scoping is exact: - a stopped app keeps the launch the activating route performs for it today: `state` answers `.notRunning` and the arm declines, so `open`-then-read still works and no macOS refusal is invented (`notRunningRefusal` stays iOS-only); - interactions keep activating: the macOS XCTest path drives events through the foreground window, and a background click would land on whatever window is on top; - a window-level screenshot keeps its raise (measured: a backgrounded `screenshotRoot(app:).screenshot()` returns the occluding window's content inside the session window's rect), but a `--fullscreen` capture of a running app no longer raises — `XCUIScreen.main` answers from any foreground, and the 0.5 s settle waits belong to the raise and are skipped with it. A stopped app keeps launching either way. The two conditions are functions the call sites read (`macReadMayBeServedInBackground`, `macAppCaptureNeedsRaise`) so the host lane pins every state they branch on. Four host-lane tests: the background read served without a raise or a fact across its three shapes, the interaction on the same app still activating, a stopped read still launching, and the raise-condition table. The canary is the arm's removal: the read test goes red with the app left foreground and a `bundle_changed` fact booked. A live run on the default backend confirms it: with Finder frontmost, `snapshot` and `get` answered from the background app, Finder stayed frontmost, and runner.log booked zero activation facts; the click after them activated with priorState=3 as it must. `help macos` states the new read behavior. Also records at the `.presentedSurface` arm why its macOS route is unreachable from the host (`alert` answers through the helper; `action-button` is refused by its owner fact before dispatch), so no second activation axis is proposed against it. Partial #3254
1 parent 24b2ce6 commit 2df8924

7 files changed

Lines changed: 236 additions & 8 deletions

File tree

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -536,12 +536,22 @@ extension RunnerTests {
536536
#else
537537
// The platform exception, written once: `SystemSurfaceHostRegistry` registers no hosts off iOS,
538538
// so nothing is ever served in place there and such a command keeps the activation route this
539-
// axis found it on.
539+
// axis found it on. The macOS host never sends one of this arm's commands here: `alert` answers
540+
// through the macOS helper (`runAppleAlert`'s host split) and `action-button` is refused by the
541+
// owner's Action Button fact (`hasAppleActionButton`) before dispatch, so no macOS request can
542+
// be activated from this arm, and only the tvOS and visionOS runners reach it off iOS.
540543
return prepareActivatedTarget(command: command)
541544
#endif
542545
case .existingApp:
543546
// No request-dependent bypass here: it decides by querying the cached target's state, and a
544547
// command that may bring nothing forward has nothing for it to settle.
548+
#if os(macOS)
549+
if macReadMayBeServedInBackground(command) {
550+
// Served in place with no binding, the same shape the iOS `.presentedSurface` arm takes:
551+
// the next command that needs the app forward pays for its own activation.
552+
return .context(ActiveCommandContext(app: resolveAppWithoutActivation(command: command)))
553+
}
554+
#endif
545555
return prepareActivatedTarget(command: command)
546556
case .mayLaunch:
547557
if shouldSkipAppActivationPreflight(command) {
@@ -775,4 +785,16 @@ extension RunnerTests {
775785
}
776786
return XCUIApplication(bundleIdentifier: bundleId)
777787
}
788+
789+
/// Whether a macOS `.existingApp` read may be served from the app exactly where it sits: the
790+
/// request names an app that is running but behind other windows. On a desktop the foreground
791+
/// repair is not a no-op for the user — it takes their frontmost app away for a command that only
792+
/// asked to look, which is what #3254 reports. The answer comes from `state`, which never launches,
793+
/// so a stopped or unknown app keeps the standing route untouched, including the launch that route
794+
/// performs; only the raise of an already-running app is in question here.
795+
@MainActor
796+
func macReadMayBeServedInBackground(_ command: Command) -> Bool {
797+
guard let bundleId = command.appBundleId?.trimmedNonEmpty else { return false }
798+
return XCUIApplication(bundleIdentifier: bundleId).state == .runningBackground
799+
}
778800
}

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift‎

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -428,18 +428,40 @@ extension RunnerTests {
428428
return Response(ok: true, data: DataPayload(text: text))
429429
case .screenshot:
430430
#if os(macOS)
431-
// macOS keeps the app-targeted capture behavior for window-level screenshots.
432-
if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
431+
// Window-level app captures raise the app; a full-screen capture of a running app and an app
432+
// already in the foreground do not (#3254).
433+
//
434+
// The raise is load-bearing for the window-level pixels, so it stays there. Measured on a macOS
435+
// host with a known window frame and a second window laid over it: for a `.runningBackground`
436+
// app `screenshotRoot(app:).screenshot()` returns that window's own rect at the right scale but
437+
// not its own content — the occluding app's windows are in the image — and the same call after
438+
// `activate()` returns the app's content. A window-level capture on this backend is a region
439+
// grab, so answering one from the background would silently report another app's screen as this
440+
// app's window. A `--fullscreen` capture is a whole-display grab that `XCUIScreen.main` answers
441+
// whichever app owns the foreground, so nothing about its pixels needs a running app raised,
442+
// and a read that asked for none takes the user's frontmost app away. A stopped app keeps being
443+
// started for either shape: there the activation is the launch it has always performed, which
444+
// this change leaves untouched. The wait belongs to the raise: only a raised app has an
445+
// animation to settle. The macOS helper serves background windows by a different mechanism,
446+
// `SCContentFilter(desktopIndependentWindow:)`, which is the native backend's path and is not
447+
// reachable here.
448+
let screenshotBundleId = command.appBundleId?.trimmedNonEmpty
449+
if let bundleId = screenshotBundleId {
433450
let targetApp = XCUIApplication(bundleIdentifier: bundleId)
434-
targetApp.activate()
451+
if macAppCaptureNeedsRaise(
452+
fullscreen: command.fullscreen,
453+
targetState: targetApp.state
454+
) {
455+
targetApp.activate()
456+
// Brief wait for the app transition animation to complete
457+
sleepFor(0.5)
458+
}
435459
activeApp = targetApp
436-
// Brief wait for the app transition animation to complete
437-
sleepFor(0.5)
438460
}
439461
let screenshot: XCUIScreenshot
440462
if command.fullscreen == true {
441463
screenshot = XCUIScreen.main.screenshot()
442-
} else if let bundleId = command.appBundleId, !bundleId.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
464+
} else if screenshotBundleId != nil {
443465
screenshot = screenshotRoot(app: activeApp).screenshot()
444466
} else {
445467
screenshot = XCUIScreen.main.screenshot()

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,18 @@ extension RunnerTests {
284284
return false
285285
}
286286

287+
/// Whether a macOS app-targeted capture must bring the app forward before answering, and so run
288+
/// the wait that settles the raise (#3254). Three exclusions, each a state or shape where the
289+
/// pixels need nothing brought forward: an app already in the foreground has nothing to raise, and
290+
/// a `--fullscreen` capture of a running app is a whole-display grab that `XCUIScreen.main`
291+
/// answers whichever app owns the foreground, so raising one for it only takes the user's own app
292+
/// away. A stopped app keeps its standing behavior either way: there the activation is also the
293+
/// launch it has always performed, which this change leaves untouched.
294+
func macAppCaptureNeedsRaise(fullscreen: Bool?, targetState: XCUIApplication.State) -> Bool {
295+
guard targetState != .runningForeground else { return false }
296+
return fullscreen != true || targetState == .notRunning
297+
}
298+
287299
@MainActor
288300
func canUseFastForegroundAppGuard(
289301
activeApp: XCUIApplication,

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,11 @@ enum CommandLaunchPolicy: Equatable {
6666
/// enforced on iOS only: `notRunningRefusal` is `#if os(iOS)`, the platform that can read an app's
6767
/// state without launching it. Off iOS these commands keep the activation route they had before
6868
/// this axis existed, and no refusal can occur there.
69+
/// On macOS the same policy also settles what happens to an app that IS running but behind other
70+
/// windows: a read is served there in place, without bringing it forward, because on a desktop the
71+
/// repair steals the user's own frontmost app for a command that only asked to look (#3254). A
72+
/// stopped macOS app keeps the launch the activating route already performs; only the raise of a
73+
/// running app was removed.
6974
case existingApp
7075
/// Brings the app forward, which bare-launches it when it is not running.
7176
case mayLaunch

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+CommandDispatchTests.swift‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -690,3 +690,132 @@ extension RunnerTests {
690690
}
691691
}
692692
#endif
693+
694+
#if AGENT_DEVICE_RUNNER_UNIT_TESTS && os(macOS)
695+
extension RunnerTests {
696+
/// Part of every macOS install, never the runner or the test host, and cheap to leave terminated.
697+
/// Its windows sit over or behind the test host's own, which is exactly the desktop shape #3254
698+
/// describes: the user's frontmost app is a different app from the session app.
699+
private static let macBackgroundTargetBundleId = "com.apple.systempreferences"
700+
private static let macForegroundStealBundleId = "com.apple.finder"
701+
702+
/// Brings Settings up, then hands the foreground to Finder, and waits until Settings reports
703+
/// itself background. `activate()` on another app is how a desktop backgrounds an app without
704+
/// quitting it, so this leaves the state a read must answer — not a launchable absence.
705+
@MainActor
706+
private func startMacTargetAndLoseTheForeground() throws -> XCUIApplication {
707+
let target = XCUIApplication(bundleIdentifier: Self.macBackgroundTargetBundleId)
708+
target.launch()
709+
XCTAssertTrue(target.waitForExistence(timeout: appExistenceTimeout))
710+
_ = XCUIApplication(bundleIdentifier: Self.macForegroundStealBundleId).activate()
711+
XCTAssertTrue(
712+
target.wait(for: .runningBackground, timeout: 10),
713+
"Finder must take the foreground and the target must settle background before the read answers from it"
714+
)
715+
return target
716+
}
717+
718+
/// The #3254 fix on the policy axis: on macOS a `.existingApp` read of a session app that is
719+
/// running behind other windows is served from where it sits. Three things must be true at once:
720+
/// the command is prepared against the app it names, no activation fact is booked (the response
721+
/// therefore discloses no repair because none happened), and the app is still background when the
722+
/// read is done — the user's frontmost app never changed. Covers the user-level read and the two
723+
/// leading reads an interaction carries (gesture viewport, selector resolution), the same shapes
724+
/// the iOS surface arm pins. Deleting the macOS arm of `.existingApp` fails every row.
725+
@MainActor
726+
func testMacReadOfABackgroundAppIsServedInBackgroundWithoutActivating() throws {
727+
let target = try startMacTargetAndLoseTheForeground()
728+
let bundleId = Self.macBackgroundTargetBundleId
729+
defer {
730+
invalidateCachedTarget(reason: "unit_test_cleanup")
731+
pendingTargetActivation = nil
732+
target.terminate()
733+
}
734+
for request in [
735+
#"{"command":"snapshot","commandId":"read","appBundleId":"\#(bundleId)"}"#,
736+
#"{"command":"gestureViewport","commandId":"read","appBundleId":"\#(bundleId)"}"#,
737+
#"{"command":"querySelector","selectorKey":"label","selectorValue":"General","commandId":"read","appBundleId":"\#(bundleId)"}"#
738+
] {
739+
invalidateCachedTarget(reason: "unit_test_setup")
740+
pendingTargetActivation = nil
741+
let command = try runnerCommandFixture(request)
742+
743+
guard case .context(let prepared) = prepareActiveCommandContext(command: command) else {
744+
return XCTFail("\(request) must be served, not refused")
745+
}
746+
XCTAssertEqual(
747+
prepared.app.state,
748+
.runningBackground,
749+
"\(request) must be served against the background app it names"
750+
)
751+
XCTAssertNil(pendingTargetActivation, "\(request) may not book an activation fact")
752+
XCTAssertEqual(
753+
target.state,
754+
.runningBackground,
755+
"\(request) may not take the user's frontmost app away"
756+
)
757+
}
758+
}
759+
760+
/// The scope limit that keeps this from being a blanket no-activate rule: an interaction against
761+
/// the same background app still comes forward through the same preparation, because on macOS the
762+
/// XCTest path drives events into the foreground window and a background click would land on
763+
/// whatever window sits on top. This is the same call with `.mayLaunch`; the two tests above and
764+
/// this one together pin the boundary the policy axis draws.
765+
@MainActor
766+
func testMacInteractionOnABackgroundAppStillActivates() throws {
767+
let target = try startMacTargetAndLoseTheForeground()
768+
defer {
769+
invalidateCachedTarget(reason: "unit_test_cleanup")
770+
pendingTargetActivation = nil
771+
target.terminate()
772+
}
773+
invalidateCachedTarget(reason: "unit_test_setup")
774+
pendingTargetActivation = nil
775+
let command = try runnerCommandFixture(
776+
#"{"command":"tap","x":10,"y":10,"commandId":"tap-bg","appBundleId":"\#(Self.macBackgroundTargetBundleId)"}"#
777+
)
778+
779+
guard case .context = prepareActiveCommandContext(command: command) else {
780+
return XCTFail("the tap must be prepared")
781+
}
782+
XCTAssertNotNil(
783+
pendingTargetActivation,
784+
"an interaction on a background app must still book its activation"
785+
)
786+
XCTAssertEqual(target.state, .runningForeground, "the tap must run against the foreground app")
787+
}
788+
789+
/// Stopped apps behave exactly as they did before this axis read macOS (#3254 correction): a read
790+
/// of a not-running session app still goes through the activating route and the launch it performs,
791+
/// because macOS has no refusal there and callers rely on `open`-then-read working. `snapshot` is
792+
/// the read whose policy the background change touched, so it is the read that proves the change is
793+
/// scoped to the running app only: the app comes up, the activation fact is booked, and nothing is
794+
/// refused. Making `macReadMayBeServedInBackground` answer `.notRunning` too would turn this
795+
/// context into a refusal and fail on the first assertion.
796+
@MainActor
797+
func testMacReadOfANotRunningAppStillLaunchesIt() throws {
798+
let target = XCUIApplication(bundleIdentifier: Self.macBackgroundTargetBundleId)
799+
target.terminate()
800+
invalidateCachedTarget(reason: "unit_test_setup")
801+
pendingTargetActivation = nil
802+
defer {
803+
invalidateCachedTarget(reason: "unit_test_cleanup")
804+
pendingTargetActivation = nil
805+
target.terminate()
806+
}
807+
let command = try runnerCommandFixture(
808+
#"{"command":"snapshot","commandId":"read","appBundleId":"\#(Self.macBackgroundTargetBundleId)"}"#
809+
)
810+
811+
guard case .context = prepareActiveCommandContext(command: command) else {
812+
return XCTFail("a read of a stopped app must keep the launch route it has always had")
813+
}
814+
XCTAssertNotEqual(target.state, .notRunning, "the stopped app must come up as it did before")
815+
XCTAssertNotNil(
816+
pendingTargetActivation,
817+
"the command that launched the app discloses the activation it paid for"
818+
)
819+
}
820+
}
821+
#endif

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+LifecycleTests.swift‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,43 @@ extension RunnerTests {
117117
}
118118
#endif
119119

120+
#if AGENT_DEVICE_RUNNER_UNIT_TESTS && os(macOS)
121+
extension RunnerTests {
122+
/// Pins every state the macOS screenshot raise condition reads (#3254): only an app that is not
123+
/// already foreground is ever raised, and only a window-level grab needs it — a `--fullscreen`
124+
/// capture of a running app answers from any foreground. A stopped app keeps its standing answer in
125+
/// both shapes, because there the activation is also the launch it has always performed; the
126+
/// change is that full-screen stops paying a raise for a RUNNING app its pixels never needed. The
127+
/// screenshot call site reads this same function, so a drifted condition is a red row rather than a
128+
/// silent behavior change.
129+
func testMacAppCaptureNeedsRaiseIsWindowLevelAndBackgroundOnly() {
130+
let foreground = XCUIApplication.State.runningForeground
131+
let background = XCUIApplication.State.runningBackground
132+
let notRunning = XCUIApplication.State.notRunning
133+
// Rows are literal expectations, not the condition recomputed: a drifted condition must land as
134+
// a red row here, not move both sides of the comparison together.
135+
let table: [(fullscreen: Bool?, state: XCUIApplication.State, expected: Bool)] = [
136+
(nil, foreground, false),
137+
(nil, background, true),
138+
(nil, notRunning, true),
139+
(false, foreground, false),
140+
(false, background, true),
141+
(false, notRunning, true),
142+
(true, foreground, false),
143+
(true, background, false),
144+
(true, notRunning, true),
145+
]
146+
for row in table {
147+
XCTAssertEqual(
148+
macAppCaptureNeedsRaise(fullscreen: row.fullscreen, targetState: row.state),
149+
row.expected,
150+
"fullscreen=\(String(describing: row.fullscreen)) state=\(row.state.rawValue)"
151+
)
152+
}
153+
}
154+
}
155+
#endif
156+
120157
#if AGENT_DEVICE_RUNNER_UNIT_TESTS
121158
extension RunnerTests {
122159
@MainActor

‎src/commands/schema/cli-help.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -739,7 +739,8 @@ Rules:
739739
Context menus are not ambient UI: secondary-click a visible target, then re-snapshot and use the new menu-item refs.
740740
Do not let iOS simulator-set scoping hide macOS desktop targets.
741741
Prefer refs/selectors over raw coordinates.
742-
macOS snapshot rects are window-space; use current refs or overlay refs instead of guessing coordinates.`,
742+
macOS snapshot rects are window-space; use current refs or overlay refs instead of guessing coordinates.
743+
On the default XCTest backend a read (snapshot, get, find) of a session app that sits behind other windows is answered from where it sits, without raising the app; interactions and a window-level screenshot of that app still bring it forward.`,
743744
},
744745
web: {
745746
summary: 'Minimal browser workflow with the managed web backend',

0 commit comments

Comments
 (0)