Skip to content

Commit 5163c8a

Browse files
kvs-coderkvs-coder
authored
feat: jsonrpc add methods - apps_launch, dump_ui
1 parent 33e663d commit 5163c8a

File tree

2 files changed

+310
-3
lines changed

2 files changed

+310
-3
lines changed

‎docs/jsonrpc_README.md‎

Lines changed: 247 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,247 @@
1+
# mobilecli JSON-RPC API
2+
3+
This document describes the JSON-RPC methods exposed by the mobilecli HTTP server at the `/rpc` endpoint.
4+
5+
Base URL: `http://<host>:<port>/rpc`
6+
7+
All requests must be JSON-RPC 2.0 objects with `jsonrpc` set to "2.0", a `method` string, an `id`, and optionally `params`.
8+
9+
Common response format:
10+
11+
```json
12+
{
13+
"jsonrpc": "2.0",
14+
"result": ... ,
15+
"error": { "code": ..., "message": ..., "data": ... },
16+
"id": <id>
17+
}
18+
```
19+
20+
Errors use standard JSON-RPC error codes and include a `data` field with additional context.
21+
22+
## Methods
23+
24+
This section documents the JSON-RPC methods registered by the server and shows example `params` and curl requests.
25+
26+
- `devices`
27+
- Description: List devices.
28+
- Params: object (optional)
29+
- `includeOffline` (bool)
30+
- `platform` (string)
31+
- `type` (string)
32+
- Example:
33+
34+
```json
35+
{"jsonrpc":"2.0","method":"devices","params":{"includeOffline":true},"id":1}
36+
```
37+
38+
- `screenshot`
39+
- Description: Take a screenshot and return base64 data.
40+
- Params: object
41+
- `deviceId` (string)
42+
- `format` (string) - `png` or `jpeg`
43+
- `quality` (int) - JPEG quality 1-100
44+
- Example:
45+
46+
```json
47+
{"jsonrpc":"2.0","method":"screenshot","params":{"deviceId":"<id>","format":"png"},"id":2}
48+
```
49+
50+
- `screencapture` (streaming)
51+
- Description: Start a screen capture stream (MJPEG or AVC). This writes a streaming response; use the HTTP `/rpc` POST to initiate streaming.
52+
- Params: object
53+
- `deviceId` (string)
54+
- `format` (string) - `mjpeg` or `avc`
55+
- `quality` (int)
56+
- `scale` (float)
57+
- Note: Progress notifications are sent inside the stream for `mjpeg`.
58+
59+
- `io_tap`
60+
- Description: Tap at coordinates.
61+
- Params: object
62+
- `deviceId` (string)
63+
- `x` (int)
64+
- `y` (int)
65+
- Example:
66+
67+
```json
68+
{"jsonrpc":"2.0","method":"io_tap","params":{"deviceId":"<id>","x":100,"y":200},"id":3}
69+
```
70+
71+
- `io_longpress`
72+
- Description: Long-press at coordinates.
73+
- Params: object
74+
- `deviceId` (string)
75+
- `x` (int)
76+
- `y` (int)
77+
- `duration` (int, ms)
78+
79+
- `io_text`
80+
- Description: Send text input.
81+
- Params: object
82+
- `deviceId` (string)
83+
- `text` (string)
84+
85+
- `io_button`
86+
- Description: Press a hardware button (HOME, VOLUME_UP, etc.).
87+
- Params: object
88+
- `deviceId` (string)
89+
- `button` (string)
90+
91+
- `io_swipe`
92+
- Description: Swipe from one point to another.
93+
- Params: object
94+
- `deviceId` (string)
95+
- `x1`,`y1`,`x2`,`y2` (ints)
96+
97+
- `io_gesture`
98+
- Description: Perform a gesture composed of actions (tap, move, wait, etc.).
99+
- Params: object
100+
- `deviceId` (string)
101+
- `actions` (array of action objects) — see `devices/wda` types in code for schema.
102+
103+
- `url`
104+
- Description: Open a URL or deep link on device.
105+
- Params: object
106+
- `deviceId` (string)
107+
- `url` (string)
108+
109+
- `device_info`
110+
- Description: Retrieve detailed device info (screen size, OS, etc.).
111+
- Params: object
112+
- `deviceId` (string)
113+
- Example:
114+
115+
```json
116+
{"jsonrpc":"2.0","method":"device_info","params":{"deviceId":"<id>"},"id":21}
117+
```
118+
119+
- `io_orientation_get`
120+
- Description: Get device orientation.
121+
- Params: object
122+
- `deviceId` (string)
123+
124+
- `io_orientation_set`
125+
- Description: Set device orientation.
126+
- Params: object
127+
- `deviceId` (string)
128+
- `orientation` (string)
129+
130+
- `device_boot`, `device_shutdown`, `device_reboot`
131+
- Description: Control device power state (boot, shutdown, reboot).
132+
- Params: object
133+
- `deviceId` (string)
134+
135+
- `dump_ui`
136+
- Description: Dump the UI tree from the device.
137+
- Params: object
138+
- `deviceId` (string)
139+
- `format` (string) - `json` (default) or `raw`
140+
141+
- `apps_launch`
142+
- Description: Launch an app on a device.
143+
- Params: object
144+
- `deviceId` (string)
145+
- `bundleId` (string) - required
146+
147+
Common notes:
148+
- For most methods `deviceId` is optional; when omitted the server auto-selects a single online device or returns an error when multiple devices are available.
149+
- Methods that interact with the UI/agent (`io_*`, `dump_ui`, `apps_launch`, `device_info`, etc.) call `StartAgent` which may start/forward WDA for iOS devices. If WDA is unresponsive the server will attempt to relaunch it.
150+
151+
## Curl examples
152+
153+
- List devices:
154+
155+
```bash
156+
curl -s -X POST http://localhost:12000/rpc \
157+
-H 'Content-Type: application/json' \
158+
-d '{"jsonrpc":"2.0","method":"devices","params":{},"id":10}'
159+
```
160+
161+
- Take a screenshot:
162+
163+
```bash
164+
curl -s -X POST http://localhost:12000/rpc \
165+
-H 'Content-Type: application/json' \
166+
-d '{"jsonrpc":"2.0","method":"screenshot","params":{"deviceId":"<id>","format":"png"},"id":11}'
167+
```
168+
169+
- Start MJPEG screen capture (streaming):
170+
171+
```bash
172+
curl -v -X POST http://localhost:12000/rpc \
173+
-H 'Content-Type: application/json' \
174+
-d '{"jsonrpc":"2.0","method":"screencapture","params":{"deviceId":"<id>","format":"mjpeg"},"id":12}'
175+
```
176+
177+
- Tap coordinates:
178+
179+
```bash
180+
curl -s -X POST http://localhost:12000/rpc \
181+
-H 'Content-Type: application/json' \
182+
-d '{"jsonrpc":"2.0","method":"io_tap","params":{"deviceId":"<id>","x":195,"y":422},"id":13}'
183+
```
184+
185+
- Long press:
186+
187+
```bash
188+
curl -s -X POST http://localhost:12000/rpc \
189+
-H 'Content-Type: application/json' \
190+
-d '{"jsonrpc":"2.0","method":"io_longpress","params":{"deviceId":"<id>","x":100,"y":200,"duration":750},"id":14}'
191+
```
192+
193+
- Send text:
194+
195+
```bash
196+
curl -s -X POST http://localhost:12000/rpc \
197+
-H 'Content-Type: application/json' \
198+
-d '{"jsonrpc":"2.0","method":"io_text","params":{"deviceId":"<id>","text":"Hello"},"id":15}'
199+
```
200+
201+
- Press button:
202+
203+
```bash
204+
curl -s -X POST http://localhost:12000/rpc \
205+
-H 'Content-Type: application/json' \
206+
-d '{"jsonrpc":"2.0","method":"io_button","params":{"deviceId":"<id>","button":"HOME"},"id":16}'
207+
```
208+
209+
- Swipe:
210+
211+
```bash
212+
curl -s -X POST http://localhost:12000/rpc \
213+
-H 'Content-Type: application/json' \
214+
-d '{"jsonrpc":"2.0","method":"io_swipe","params":{"deviceId":"<id>","x1":100,"y1":200,"x2":300,"y2":400},"id":17}'
215+
```
216+
217+
- Open URL:
218+
219+
```bash
220+
curl -s -X POST http://localhost:12000/rpc \
221+
-H 'Content-Type: application/json' \
222+
-d '{"jsonrpc":"2.0","method":"url","params":{"deviceId":"<id>","url":"https://example.com"},"id":18}'
223+
```
224+
225+
- Get device info:
226+
227+
```bash
228+
curl -s -X POST http://localhost:12000/rpc \
229+
-H 'Content-Type: application/json' \
230+
-d '{"jsonrpc":"2.0","method":"device_info","params":{"deviceId":"<id>"},"id":21}'
231+
```
232+
233+
- Dump UI (json):
234+
235+
```bash
236+
curl -s -X POST http://localhost:12000/rpc \
237+
-H 'Content-Type: application/json' \
238+
-d '{"jsonrpc":"2.0","method":"dump_ui","params":{"deviceId":"<id>","format":"json"},"id":19}'
239+
```
240+
241+
- Launch app:
242+
243+
```bash
244+
curl -s -X POST http://localhost:12000/rpc \
245+
-H 'Content-Type: application/json' \
246+
-d '{"jsonrpc":"2.0","method":"apps_launch","params":{"deviceId":"<id>","bundleId":"com.example.app"},"id":20}'
247+
```

‎server/server.go‎

Lines changed: 63 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,10 @@ func handleJSONRPC(w http.ResponseWriter, r *http.Request) {
184184
result, err = handleDeviceShutdown(req.Params)
185185
case "device_reboot":
186186
result, err = handleDeviceReboot(req.Params)
187+
case "dump_ui":
188+
result, err = handleDumpUI(req.Params)
189+
case "apps_launch":
190+
result, err = handleAppsLaunch(req.Params)
187191
case "":
188192
err = fmt.Errorf("'method' is required")
189193

@@ -455,6 +459,16 @@ type DeviceRebootParams struct {
455459
DeviceID string `json:"deviceId"`
456460
}
457461

462+
type DumpUIParams struct {
463+
DeviceID string `json:"deviceId"`
464+
Format string `json:"format,omitempty"` // "json" or "raw"
465+
}
466+
467+
type AppsLaunchParams struct {
468+
DeviceID string `json:"deviceId"`
469+
BundleID string `json:"bundleId"`
470+
}
471+
458472
func handleIoButton(params json.RawMessage) (interface{}, error) {
459473
if len(params) == 0 {
460474
return nil, fmt.Errorf("'params' is required with fields: deviceId, button")
@@ -666,6 +680,52 @@ func handleDeviceReboot(params json.RawMessage) (interface{}, error) {
666680
return response.Data, nil
667681
}
668682

683+
func handleDumpUI(params json.RawMessage) (interface{}, error) {
684+
if len(params) == 0 {
685+
return nil, fmt.Errorf("'params' is required with fields: deviceId")
686+
}
687+
688+
var dumpUIParams DumpUIParams
689+
if err := json.Unmarshal(params, &dumpUIParams); err != nil {
690+
return nil, fmt.Errorf("invalid parameters: %v. Expected fields: deviceId, format (optional)", err)
691+
}
692+
693+
req := commands.DumpUIRequest{
694+
DeviceID: dumpUIParams.DeviceID,
695+
Format: dumpUIParams.Format,
696+
}
697+
698+
response := commands.DumpUICommand(req)
699+
if response.Status == "error" {
700+
return nil, fmt.Errorf("%s", response.Error)
701+
}
702+
703+
return response.Data, nil
704+
}
705+
706+
func handleAppsLaunch(params json.RawMessage) (interface{}, error) {
707+
if len(params) == 0 {
708+
return nil, fmt.Errorf("'params' is required with fields: deviceId, bundleId")
709+
}
710+
711+
var appsLaunchParams AppsLaunchParams
712+
if err := json.Unmarshal(params, &appsLaunchParams); err != nil {
713+
return nil, fmt.Errorf("invalid parameters: %v. Expected fields: deviceId, bundleId", err)
714+
}
715+
716+
req := commands.AppRequest{
717+
DeviceID: appsLaunchParams.DeviceID,
718+
BundleID: appsLaunchParams.BundleID,
719+
}
720+
721+
response := commands.LaunchAppCommand(req)
722+
if response.Status == "error" {
723+
return nil, fmt.Errorf("%s", response.Error)
724+
}
725+
726+
return response.Data, nil
727+
}
728+
669729
func sendJSONRPCError(w http.ResponseWriter, id interface{}, code int, message string, data interface{}) {
670730
response := JSONRPCResponse{
671731
JSONRPC: "2.0",
@@ -777,9 +837,9 @@ func handleScreenCapture(w http.ResponseWriter, params json.RawMessage) error {
777837

778838
// Start screen capture and stream to the response writer
779839
err = targetDevice.StartScreenCapture(devices.ScreenCaptureConfig{
780-
Format: screenCaptureParams.Format,
781-
Quality: quality,
782-
Scale: scale,
840+
Format: screenCaptureParams.Format,
841+
Quality: quality,
842+
Scale: scale,
783843
OnProgress: progressCallback,
784844
OnData: func(data []byte) bool {
785845
_, writeErr := w.Write(data)

0 commit comments

Comments
 (0)