Skip to content

Commit fc59620

Browse files
authored
feat(action-button): press the iPhone Action Button (callstack#2702)
* feat(action-button): press the iPhone Action Button Add `agent-device action-button` and `client.command.actionButton()`: one press of the iPhone/iPad Action Button, shaped like `home` rather than `tv-remote`. XCUITest exposes `XCUIDevice.press(.action)` with no hold duration, so there is no `long-press` and no `--duration-ms`, and the slide surface on the same edge belongs to Camera Control, which this command does not drive. The press is declared as a runner-lifecycle command on the Swift side so the app-activation preflight cannot foreground the app the press is meant to leave alone, and no post-action observation is declared. Which model actually carries the button is not in `DeviceInfo`, so the Apple fact claims the iPhone/iPad leaf and the runner answers for the hardware with `hasHardwareButton(.action)`, reporting a typed `UNSUPPORTED_OPERATION`. Every other platform states its own refusal cell. Verified on iOS Simulators: an iPhone 17 Pro press returns `Pressed Action Button` with no runner activation for that command, and an iPhone SE (3rd gen) refuses with `UNSUPPORTED_OPERATION` on both the CLI and the typed client, repeatably and without invalidating the session. Closes callstack#2699 * chore(gates): register action-button in the exports snapshot and coverage tables The new export specifier, the six-platform coverage declaration and its evidence rows, the per-platform classification counts, the generic-route catalog, the runtime-binding conformance table, and the provider output guard's driven set all enumerate the public surface, so each one names `action-button`. * fix(action-button): keep the recorded-failure conversion on the press `isLifecycle` does two things: it skips the app-activation preflight, and it exempts the command from turning a recorded XCTest issue into `XCTEST_RECORDED_FAILURE`. The press joined the lifecycle group for the first and silently took the second, which matters most here because the press declares no settle and no post-action observation, so that conversion was the only evidence it landed. The press now classifies like `home` and skips activation through `shouldSkipAppActivationPreflight`, where `.alert` already exempts itself for the same reason. `pressActionButton` also refuses below iOS 16 instead of pressing a control that API cannot express there, and two Swift tests pin both decisions where they are decided. * refactor(action-button): declare the press through the shared zero-argument group The press is a zero-argument interactor operation, which is a shape the contracts already had twice over. Rather than a third module, a package subpath, and a facts helper, `NoArgumentInteractorOperations` is declared with the other operations and bound by `bindNoArgumentInteractorOperations` next to the two interactor sources; owners state their `actionButton` cell the way they state `ensureReady`, and the catalog gains a row, not an import. That removes the four eager-closure growths this PR caused and the extra export surface. Two follow-ups from the same review: the provider fixture now reads the kernel rule the Apple fact reads instead of restating the iOS/iPadOS compare, and the runner protocol fixture test checks itself against the trait manifest — the exhaustive runner-command declaration — instead of a second hand-maintained list that had already pushed the file past its size tripwire. * docs(action-button): claim only what a simulator can prove The description told agents that an assigned Shortcut or App Intent fires whatever the app's state. Simulators run neither, so nothing in this PR can show that; the command proves the press was dispatched and that the session app was not brought forward. The help text, `commands.md`, and a CHANGELOG entry now say exactly that, and name a physical iPhone as the only place delivery is verifiable. * chore(gates): drop the contracts action-button export subpath The generated contracts export snapshot returns to its merge-base enumeration now that the press declares itself through an export the closures already evaluate. * refactor(contracts): keep the interactor binding module a leaf Layering R10 counts the largest type-level cycle, and a type-only import of `platform-runtime-operations` from `interactor-operation-binding` pulled the six interactor-backed runtime modules into it (baseline zero). The catalog already imports that type, so the zero-argument group's binder moves there — the one module that already knows both the operations catalog and the interactor sources — and the binding module keeps its two imports. This commit supersedes the binding placement in 267c5c0, which was never pushed.
1 parent 3155aea commit fc59620

76 files changed

Lines changed: 648 additions & 57 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,15 @@
1010
second remote stop. The capture is always the whole simulator or emulator screen; `--fps` and
1111
`--hide-touches` are refused, and a recording cannot be reattached after a daemon restart. The
1212
web recorder now shares the same transport-recording runtime.
13+
- Added (ios): `action-button` presses the iPhone or iPad Action Button once, and `client.command.actionButton()`
14+
does the same. XCUITest exposes `XCUIDevice.press(.action)` with no hold-duration overload, so there is no
15+
`long-press` and no `--duration-ms`, and the slide surface on the same edge is Camera Control, which this
16+
command does not drive. The press is dispatched without the runner's app-activation preflight and with no
17+
post-action observation, so the session app keeps the state the press found; the runner asks the device for
18+
the button with `hasHardwareButton(.action)` and refuses with `UNSUPPORTED_OPERATION` on a model that has
19+
none, and Android, HarmonyOS, Vega, Linux, web, tvOS, macOS, and visionOS each state their own refusal.
20+
Simulators run no Shortcuts or App Intents, so what a press triggers is verifiable only on a physical
21+
iPhone (#2699).
1322
- Fixed (android): `clipboard read` and `clipboard write` stop reporting success on a build whose
1423
clipboard service has no shell command. Android 16 (API 36) answers every `adb shell cmd clipboard …`
1524
with the framework default `Binder.handleShellCommand` — `No shell command implementation.` on

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -521,6 +521,23 @@ extension RunnerTests {
521521
)
522522
}
523523

524+
func testXCTestRecordedFailureResponseFailsActionButtonSuccess() throws {
525+
// The Action Button press carries no settle and no post-action observation, so this conversion is
526+
// the only evidence the press landed. That is why the press is not classified runner-lifecycle:
527+
// `isLifecycle` would silence the conversion here (#2699, #2702 review).
528+
let command = try runnerCommandFixture(#"{"command":"actionButton","commandId":"action-button-1"}"#)
529+
let response = Response(ok: true, data: DataPayload(message: "actionButton"))
530+
531+
let failureResponse = xctestRecordedFailureResponse(command: command, response: response)
532+
533+
XCTAssertEqual(failureResponse?.ok, false)
534+
XCTAssertEqual(failureResponse?.error?.code, "XCTEST_RECORDED_FAILURE")
535+
XCTAssertEqual(
536+
failureResponse?.error?.message,
537+
"XCTest recorded a failure while executing actionButton; the action may not have been performed."
538+
)
539+
}
540+
524541
func testXCTestRecordedFailureResponseDoesNotWrapReadOnlyOrRunnerFatalResponses() throws {
525542
let snapshotCommand = try runnerCommandFixture(#"{"command":"snapshot","commandId":"snapshot-1"}"#)
526543
let tapCommand = try runnerCommandFixture(#"{"command":"tap","commandId":"tap-1"}"#)
@@ -610,6 +627,17 @@ extension RunnerTests {
610627
XCTAssertFalse(shouldSkipAppActivationPreflight(coordinateTap))
611628
}
612629

630+
func testActionButtonPressSkipsAppActivationPreflightWithoutBeingRunnerLifecycle() throws {
631+
currentApp = nil
632+
currentBundleId = nil
633+
let press = try runnerCommandFixture(#"{"command":"actionButton","commandId":"action-button-1"}"#)
634+
635+
// The skip is its own decision, reached without the lifecycle flag that would also drop the
636+
// recorded-failure conversion; no cached target and no foreground app is required for it.
637+
XCTAssertFalse(isRunnerLifecycleCommand(.actionButton))
638+
XCTAssertTrue(shouldSkipAppActivationPreflight(press))
639+
}
640+
613641
func testPrepareActiveCommandContextRoutesBlockingSystemModalToSpringboard() throws {
614642
blockingSystemModalPresenceOverrideForTesting = true
615643
currentApp = nil
@@ -2080,6 +2108,18 @@ extension RunnerTests {
20802108
case .appSwitcher:
20812109
performAppSwitcherGesture(app: activeApp)
20822110
return Response(ok: true, data: DataPayload(message: "appSwitcher"))
2111+
case .actionButton:
2112+
guard pressActionButton() else {
2113+
return Response(
2114+
ok: false,
2115+
error: ErrorPayload(
2116+
code: "UNSUPPORTED_OPERATION",
2117+
message: "actionButton requires a device model with an Action Button",
2118+
hint: "The Action Button is on iPhone 15 Pro and later and iPad Pro (M4) and later. Assign a Shortcut or App Intent to it in Settings > Action Button."
2119+
)
2120+
)
2121+
}
2122+
return Response(ok: true, data: DataPayload(message: "actionButton"))
20832123
case .keyboardDismiss:
20842124
let result = dismissKeyboard(app: activeApp)
20852125
if result.wasVisible && !result.dismissed {
@@ -2540,6 +2580,14 @@ extension RunnerTests {
25402580
if command.command == .alert {
25412581
return true
25422582
}
2583+
// A hardware Action Button press belongs to the system, not to the session app: the Shortcut or
2584+
// App Intent behind it is expected to run whether that app is foregrounded, backgrounded, or
2585+
// terminated, and activating first would foreground exactly what the press should leave alone.
2586+
// The press keeps its recorded-failure conversion, which `isLifecycle` would have removed
2587+
// (#2699, #2702 review).
2588+
if command.command == .actionButton {
2589+
return true
2590+
}
25432591
// Coordinate-only synthesized taps can run after an AX-fatal foreground screen because they do not
25442592
// need app activation, window lookup, keyboard lookup, or element resolution. Selector/text
25452593
// interactions intentionally stay on the normal AX path because they need an element query.

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandJournal.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,7 @@ final class RunnerCommandJournal {
137137
return false
138138
case .tap, .mouseClick, .longPress, .drag,
139139
.remotePress, .type, .swipe, .scroll, .desktopScroll, .findText, .querySelector, .readText, .back,
140-
.backInApp, .backSystem, .home, .rotate, .appSwitcher, .keyboardDismiss, .keyboardReturn,
140+
.backInApp, .backSystem, .home, .rotate, .appSwitcher, .actionButton, .keyboardDismiss, .keyboardReturn,
141141
.alert, .sequence, .gesture, .gestureViewport, .recordStart, .recordStop,
142142
.status, .uptime, .activate, .terminate, .targetReset, .shutdown:
143143
return true

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,24 @@ extension RunnerTests {
124124
#endif
125125
}
126126

127+
/// Presses the iPhone Action Button, returning false when this device cannot express the press.
128+
///
129+
/// `XCUIDevice.press(_:)` has no hold-duration overload, so a single press is the whole gesture
130+
/// this API can express. `hasHardwareButton(.action)` is what separates a model with the button
131+
/// from one without, and both it and `Button.action` need iOS 16 while the deployment target is
132+
/// lower, so an older system refuses rather than pressing a control that cannot exist there.
133+
@discardableResult
134+
func pressActionButton() -> Bool {
135+
#if os(iOS)
136+
guard #available(iOS 16.0, *) else { return false }
137+
guard XCUIDevice.shared.hasHardwareButton(.action) else { return false }
138+
XCUIDevice.shared.press(.action)
139+
return true
140+
#else
141+
return false
142+
#endif
143+
}
144+
127145
func findElement(app: XCUIApplication, text: String) -> XCUIElement? {
128146
let predicate = NSPredicate(format: "label CONTAINS[c] %@ OR identifier CONTAINS[c] %@ OR value CONTAINS[c] %@", text, text, text)
129147
let element = app.descendants(matching: .any).matching(predicate).firstMatch

‎apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ enum CommandType: String, Codable {
2323
case home
2424
case rotate
2525
case appSwitcher
26+
case actionButton
2627
case keyboardDismiss
2728
case keyboardReturn
2829
case alert
@@ -97,6 +98,14 @@ extension CommandType {
9798
case .recordStop, .uptime, .terminate, .targetReset, .shutdown:
9899
return CommandTraits(isInteraction: false, readOnly: .never, isLifecycle: true)
99100

101+
// A hardware press mutates, is not an element interaction, and is not runner-lifecycle. It stays
102+
// outside the lifecycle group because that flag also exempts a command from the recorded-failure
103+
// conversion, and this command has no settle or post-action observation, so that conversion is
104+
// the only evidence the press landed. It skips the app-activation preflight on its own terms in
105+
// `shouldSkipAppActivationPreflight`, the way `.alert` does (#2699, #2702 review).
106+
case .actionButton:
107+
return CommandTraits(isInteraction: false, readOnly: .never, isLifecycle: false)
108+
100109
case .status:
101110
return CommandTraits(isInteraction: false, readOnly: .always, isLifecycle: true)
102111

‎packages/command-registry/src/__tests__/command-result.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,7 @@ test('CommandResultMap is seeded only from already-existing contract result type
150150
| 'back'
151151
| 'orientation'
152152
| 'app-switcher'
153+
| 'action-button'
153154
| 'clipboard'
154155
| 'appstate'
155156
| 'keyboard'

‎packages/command-registry/src/command-result.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import type {
1010
TriggerAppEventCommandResult,
1111
} from '@agent-device/contracts/device';
1212
import type {
13+
ActionButtonCommandResult,
1314
AppSwitcherCommandResult,
1415
BackCommandResult,
1516
HomeCommandResult,
@@ -56,6 +57,7 @@ import type { ReplayCommandResult, ReplaySuiteResult } from '@agent-device/contr
5657
* re-read of the handler's literal return; see the per-type docstrings.
5758
*/
5859
export interface CommandResultMap {
60+
'action-button': ActionButtonCommandResult;
5961
'app-switcher': AppSwitcherCommandResult;
6062
appstate: AppStateCommandResult;
6163
back: BackCommandResult;

‎packages/command-registry/src/registry.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import { networkDumpUse } from '@agent-device/contracts/network-runtime-plan';
3939
import { inventoryUse } from '@agent-device/contracts/platform-module';
4040
import {
4141
alertRuntimePlanUses,
42+
actionButtonRuntimeUse,
4243
appEventRuntimeUse,
4344
appStateRuntimeUses,
4445
appSwitcherRuntimeUse,
@@ -1524,6 +1525,19 @@ export const RAW_COMMAND_DESCRIPTORS = [
15241525
batchable: true,
15251526
platformExecution: { kind: 'device-runtime', uses: [appSwitcherRuntimeUse] },
15261527
},
1528+
{
1529+
name: 'action-button',
1530+
...(ownerFilesEnabled ? { ownerFiles: ['src/commands/system/index.ts'] as const } : {}),
1531+
catalog: { group: 'public', key: 'actionButton' },
1532+
frameworkTier: 'extended',
1533+
// Admission is the owner's `actionButton` fact, the same ADR 0019 §9 shape as `home`. The
1534+
// generic mutating traits are load-bearing beyond their shared shape: no post-action
1535+
// observation is declared, because the press is delivered to the system rather than to the
1536+
// session app, and settling would re-capture (and so foreground) the app the press is supposed
1537+
// to leave alone (#2699).
1538+
...GENERIC_MUTATING_COMMAND_TRAITS,
1539+
platformExecution: { kind: 'device-runtime', uses: [actionButtonRuntimeUse] },
1540+
},
15271541
{
15281542
name: 'install-from-source',
15291543
deviceClaimPolicy: 'none',

‎packages/contracts/src/client-system.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,8 @@ export type OrientationCommandOptions = DeviceCommandBaseOptions & {
9292

9393
export type AppSwitcherCommandOptions = DeviceCommandBaseOptions;
9494

95+
export type ActionButtonCommandOptions = DeviceCommandBaseOptions;
96+
9597
export type TvRemoteCommandOptions = DeviceCommandBaseOptions & {
9698
button: TvRemoteButton;
9799
durationMs?: number;

‎packages/contracts/src/facades/client.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,7 @@ export type { PermissionTarget, SettingsUpdateOptions } from '../client-settings
112112
export type {
113113
AlertCommandOptions,
114114
AppStateCommandOptions,
115+
ActionButtonCommandOptions,
115116
AppSwitcherCommandOptions,
116117
BackCommandOptions,
117118
ClipboardCommandOptions,

0 commit comments

Comments
 (0)