fix(ios): guard foldable capture and transposed window coordinates - #1624

…iPhone (#1664)

## Summary

- Add a `touchPaths` command to the iOS device runner: one timed
waypoint path per finger (1 to 5), played at once as a single event
record through XCTest's event synthesizer.
- `gesture-pinch`, `gesture-rotate` and `gesture-custom` now run on a
physical iPhone through it. e2e `pinch`/`rotate` steps no longer fail
upfront there.
- A plain Down/Up `gesture-custom` (press-hold, drag, long-press pickup)
stays on `XCUICoordinate`, whose hold semantics a pickup needs. Only a
second finger or `Move` waypoints take the new path.
- Update the docs, rules and skills that said physical iPhones have no
two-finger gestures, and drop the same line from `describe`'s header.

## Why

On hardware the runner could only draw one straight press-then-drag, so
an agent testing an app with a pinch-to-zoom view, a map, a canvas, or
any custom multi-touch handler had to give up or drive the app's zoom
buttons. The docs and tool descriptions say "XCTest exposes no
two-finger coordinate API". That is true of the public `XCUICoordinate`
API, but XCTest builds its own touches from `XCPointerEventPath` and
`XCSynthesizedEventRecord`, and WebDriverAgent has used them for W3C
multi-finger actions for years.

## How it works

- `ArgentTouchSynthesizer` (Objective-C) looks both classes up with
`NSClassFromString` and checks every selector with
`instancesRespondToSelector:` before use. Nothing links against a
private symbol.
- If an Xcode renames or drops one, `touchPaths` answers
`UNSUPPORTED_OPERATION` with the name of what is missing. Every other
command keeps working, and single-finger gestures are unaffected.
- The runner is a test bundle installed on the developer's own phone and
never ships through the App Store, so the private API is a maintenance
risk but not a review risk. I would like your view on whether that trade
is acceptable. There is no public route to a second finger.
- Wire shape: `fingers: [{ points: [{ x, y, tMs }] }]`, absolute points
in the app's space like `tap` and `drag`. A finger whose first waypoint
is later than 0 touches down late, which is how a second finger joins a
gesture in progress. Documented in `PROTOCOL.md`.
- Validation is on the runner: 1 to 5 fingers, finite values, times
within 0 to 30 000 ms and non-decreasing per path.
- Pinch and rotate geometry moved from the tools into `gesture-utils` so
Harmony, the simulator loops and the iPhone path share one
implementation. Frames are 16 ms apart, the cadence the simulator and
emulator paths already use.
- The event record is built with
`initWithName:displayID:interfaceOrientation:` when XCTest and the main
screen offer a display id, and with the older
`initWithName:interfaceOrientation:` otherwise. Either one passes the
availability check, so an Xcode that keeps only one of them still works.
- Only an app drawn in **portrait** is accepted. The check reads the
app's own interface orientation (`XCUIApplication`'s
`interfaceOrientation`), not the way the phone is held, so a
portrait-only app on a phone turned sideways is not refused. An app
drawn in landscape is refused because I have not checked landscape on
hardware, and a touch that lands somewhere else without an error is
worse than a refusal. An orientation the runner cannot read is refused
too, rather than guessed. Lifting this needs a point mapping into the
panel's portrait space: callstack/agent-device#3838 has one, and I can
add it once someone has verified landscape on a phone.

## Verified on hardware

iPhone 12, iOS 27.0.1, Xcode 27.1, driven through the tool-server built
from this branch with `ARGENT_IOS_RUNNER_PROJECT` pointing at it, Safari
on a Wikipedia article. There were two runs, because the third commit
changed which record initializer is used.

**First two commits** (`initWithName:interfaceOrientation:`):

| Call | Result |
| --- | --- |
| `gesture-pinch` 0.1 to 0.6 over 400 ms | page zoomed about 3x |
| `gesture-pinch` 0.6 to 0.1 | zoomed back out |
| `gesture-rotate` | returns `rotated: true` (Safari has no rotate
handler, so there is nothing visible) |
| `gesture-custom`, one finger, 3 `Move` waypoints | page scrolled |
| `gesture-custom`, second finger joining late | page zoomed further |
| `gesture-custom`, plain Down/Up | still takes the `drag` path |
| `gesture-custom`, train starting with `Move` | refused before touching
the device |

**Third commit** (`initWithName:displayID:interfaceOrientation:`,
display id 1, orientation read as portrait). A temporary log line,
removed before committing, confirmed both values. I then re-ran on the
committed code:

| Call | Result |
| --- | --- |
| `gesture-custom`, four-event train at the off-centre "Bell Labs" link
(takes `touchPaths`, not `drag`) | opened the Bell Labs article, so the
touch landed where it was sent |
| `gesture-pinch` out, then in | zoomed, then back |
| `gesture-rotate` | returns `rotated: true` |
| `gesture-custom`, train starting with `Move` | refused before touching
the device |

I confirmed the synthesizer in a standalone XCUITest before wiring it
into the runner.

**Not verified:** other Xcode or iOS versions, iPads (not supported by
the runner), landscape (refused), a portrait-only app on a phone held
sideways (accepted by the check but not tried), any app other than
Safari, and timing fidelity. In the standalone probe a 0.6 s script took
0.96 s end to end, so expect gestures to run somewhat longer than their
nominal duration.

## Tests

- `ios-device-touch-paths.test.ts` (new): exact waypoints for pinch
(spread, odd duration, angle plus drifting centre, off-screen clamp),
rotate, and trains (delay spacing, late and early second finger,
refusals), plus the `touchPaths` wire shape and the pinch/rotate tools
on a device.
- `gesture-custom.test.ts`: the "rejects what XCTest cannot replay"
cases are replaced with the new behaviour and the remaining refusals.
- `flow-pinch` / `flow-rotate` / `describe-format-tree` / capability
ratchet updated for the new behaviour.
- `npm run build`, `npm run knip` and eslint on the changed files pass.
The full tool-server suite has 3 failures that are identical on an
untouched `main` in my environment (two real-ffmpeg letterbox tests and
the `e2e-tool-aliases` listing test), so I left them alone.
- There is no unit-test harness for the Swift side, so the runner
changes are covered by the hardware runs above.

## Related

- #944 (merged) listed this as a known gap: "gesture-pinch,
gesture-rotate: no two-finger coordinate API on hardware", and made the
e2e pinch/rotate steps fail upfront. This PR lifts that limit and
removes the gate. I found no open issue or PR for it.
- #538 (closed) wanted multi-touch on hardware through CoreDevice and
noted it might need XCTest instead. This is the XCTest route.
- callstack/agent-device#3838 added the same capability to
agent-device's XCTest runner through the same private classes, verified
on an iPhone 12. The third commit follows its choice of record
initializer and its use of the app's interface orientation.
- #489 (draft, no activity since 10 Jul) proposed pinch and rotate
through WebDriverAgent, before the XCUITest runner existed. This is the
runner-native equivalent.
- #1528 and #1624 also edit `ArgentRunnerSession+Gestures.swift` and
`RunnerProtocol.swift` (#1528 also `PROTOCOL.md` and
`runner-commands.ts`). The features are independent, but expect small
textual conflicts in whichever merges second. #1624 changes how window
coordinates are mapped on foldables, so it is worth a look alongside
this PR's portrait-only rule.
- Out of scope: Android. Physical-Android two-finger gestures are
blocked by #1628 / #1645, which live in the closed-source
`simulator-server`.

---------

Co-authored-by: Hubert Gancarczyk <[email protected]>