iOS Simulator record can return a valid MP4 of Agent Device Runner after target app open fails #1162

Summary

On [email protected], I reproduced an iOS Simulator recording case where the target app failed to open, but record start and record stop still returned a playable H.264 MP4.

The requested target app was Settings. A sampled frame from the MP4 shows Agent Device Runner.

This can create false-positive visual proof for AI-agent workflows: the video file is valid, but it captures the wrong surface.

Version

agent-device --version

Observed:

0.19.0

Repro

1. Have another agent-device session own the same iOS Simulator.

2. Run:

agent-device open Settings --platform ios --device "iPhone 16" --session ad-rec-repro --relaunch

Observed:

Error (DEVICE_IN_USE): Device is already in use by session "<existing-session>".

3. Continue with recording in the failed session name:

agent-device record start ./ios-nav-default.mp4 --platform ios --device "iPhone 16" --session ad-rec-repro --max-size 720 --quality medium
agent-device record stop --platform ios --device "iPhone 16" --session ad-rec-repro

Observed: both commands returned the MP4 output path.

4. Validate the MP4:

ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,width,height,avg_frame_rate,nb_frames,duration -of json ./ios-nav-default.mp4

Observed:

{
  "codec_name": "h264",
  "width": 220,
  "height": 480,
  "avg_frame_rate": "30/1",
  "duration": "17.566667",
  "nb_frames": "527"
}

5. Extract a frame:

ffmpeg -v error -ss 8.78 -i ./ios-nav-default.mp4 -frames:v 1 ./ios-nav-default-sample.png -y

Observed: the sampled frame shows Agent Device Runner, not Settings.

Actual result

record start and record stop returned a valid H.264 MP4 after open Settings failed with DEVICE_IN_USE.

The video captured Agent Device Runner.

Expected result

For iOS Simulator recording, record start should fail when the selected session has no active target app, unless the caller explicitly opts into whole-simulator / record-only capture.

Why this matters

Agent workflows use recordings as visual proof. A valid MP4 of the wrong surface is worse than a failed command because it can be mistaken for successful verification.

Requested changes

1. Fail iOS Simulator record start when the selected session has no active app session, unless an explicit flag is passed.
2. Add JSON fields to record start and record stop:
   • requested app / bundle
   • active session app / bundle
   • foreground app / bundle at start
   • foreground app / bundle at stop
   • recording backend, e.g. simctl recordVideo, adb screenrecord, runner AVAssetWriter
   • recordOnlySession
   • duration
   • dimensions
   • frame count when available
   • chunk count and chunk paths
   • showTouches
   • overlay status
   • warnings
3. Warn or fail when the foreground app is Agent Device Runner.

Mechanism note

I inspected the installed package and confirmed the iOS Simulator backend calls:

xcrun simctl io <device-id> recordVideo <outPath>

That means a playable MP4 proves the simulator display was recorded; it does not prove the requested app was foregrounded.