|
| 1 | +# @agent-device/webos |
| 2 | + |
| 3 | +Drive web apps on LG webOS TVs with [agent-device](https://agent-device.dev). The plugin launches |
| 4 | +the app with the webOS CLI, opens the `ares-inspect` tunnel, and drives the page over the Chrome |
| 5 | +DevTools Protocol: remote keys, pointer, text, snapshots, screenshots, console and network. System |
| 6 | +buttons (Home, volume) go to the TV as real remote presses over SSAP, LG's remote protocol. |
| 7 | + |
| 8 | +Status: experimental. Verified on an LG TV (webOS, dev build of a React Native Web app): |
| 9 | +`connect`, `open`, `snapshot -i`, `tv-remote` (directions, select, back, volume, home), `home`, |
| 10 | +`is focused`, `press @ref --settle`, `wait text`, `find`, `screenshot`, `logs`, `network dump`, |
| 11 | +`perf memory --json`, `apps`, `appstate`, `install`, `uninstall`, `close`. |
| 12 | + |
| 13 | +## How it plugs in |
| 14 | + |
| 15 | +agent-device leases only iOS, Android and HarmonyOS devices, so a webOS TV enters as an |
| 16 | +**Android TV device that this plugin owns** (`platform: android`, `target: tv`, id |
| 17 | +`webos:<ares device>`). Every command on that device goes to this plugin's runtime, never to adb. |
| 18 | + |
| 19 | +## Requirements |
| 20 | + |
| 21 | +- webOS CLI (`ares-*`) on `PATH`, or the directory that holds it passed as `aresBin`; see LG's |
| 22 | + [webOS CLI installation](https://webostv.developer.lge.com/develop/tools/cli-installation) |
| 23 | +- The TV in Developer Mode and registered with `ares-setup-device`; see LG's |
| 24 | + [App Testing with Developer Mode App](https://webostv.developer.lge.com/develop/getting-started/developer-mode-app) |
| 25 | +- The app installed on the TV |
| 26 | +- The TV and this host on the same network |
| 27 | +- For system buttons and `appstate`: SSAP pairing (see below) |
| 28 | + |
| 29 | +## Install |
| 30 | + |
| 31 | +```sh |
| 32 | +agent-device plugins add @agent-device/webos |
| 33 | +agent-device daemon stop # the next command starts a daemon with the plugin loaded |
| 34 | +``` |
| 35 | + |
| 36 | +Plugin options: `aresBin` (directory with the `ares-*` CLIs), `device` (default ares device name), |
| 37 | +`ssapKeyFile` (where SSAP pairing keys are kept, default `webos-ssap-keys.json` in the |
| 38 | +agent-device home). |
| 39 | + |
| 40 | +To try a local build, pack the package and install the tarball into a plugin store installation: |
| 41 | + |
| 42 | +```sh |
| 43 | +pnpm --filter @agent-device/webos pack --out /tmp/agent-device-webos.tgz |
| 44 | +installation=$(uuidgen) |
| 45 | +npm install --prefix ~/.agent-device/plugins/$installation /tmp/agent-device-webos.tgz |
| 46 | +``` |
| 47 | + |
| 48 | +Then register `{"plugins": {"@agent-device/webos": {"installation": "<installation>"}}}` in |
| 49 | +`~/.agent-device/config.json` and run `agent-device daemon stop`. |
| 50 | + |
| 51 | +### SSAP pairing |
| 52 | + |
| 53 | +The first system-button press or `appstate` on a TV shows a pairing prompt on screen. Someone at |
| 54 | +the TV accepts it with the remote within 60 s. The client key is then stored per TV IP in |
| 55 | +`ssapKeyFile`, and later presses connect without a prompt. A new IP prompts again. |
| 56 | + |
| 57 | +## Use |
| 58 | + |
| 59 | +```sh |
| 60 | +agent-device daemon stop # the lease TTL is read when the daemon starts |
| 61 | +export AGENT_DEVICE_LEASE_TTL_MS=1800000 AGENT_DEVICE_LEASE_MAX_TTL_MS=1800000 # see Known limitations |
| 62 | +agent-device connect webos [--device <ares device>] # defaults to the ares default device |
| 63 | +agent-device open <appId> # ares-launch, then attach over CDP |
| 64 | +agent-device tv-remote press down |
| 65 | +agent-device snapshot -i |
| 66 | +agent-device screenshot ./tv.png |
| 67 | +agent-device logs start # then logs path / network dump |
| 68 | +agent-device close <appId> |
| 69 | +agent-device disconnect |
| 70 | +``` |
| 71 | + |
| 72 | +Later commands use the session `connect` created; `connect` prints its name for `--session`. |
| 73 | + |
| 74 | +## Supported |
| 75 | + |
| 76 | +| Command | How | |
| 77 | +| --- | --- | |
| 78 | +| `open` / `close` | `ares-launch` / `ares-launch --close`, then the `ares-inspect` tunnel | |
| 79 | +| `tv-remote press up/down/left/right/select/back`, `back` | `Input.dispatchKeyEvent` with the LG remote codes (Back = 461) | |
| 80 | +| `tv-remote press home/menu/volume-up/volume-down/mute/channel-up/channel-down/media-rewind/media-fast-forward`, `home` | SSAP pointer-socket button presses | |
| 81 | +| `appstate` | SSAP `getForegroundAppInfo` | |
| 82 | +| `apps`, `install <appId> <ipk>`, `uninstall <appId>` | `ares-install --listfull` / `ares-install` / `ares-install --remove` | |
| 83 | +| `logs` | console calls, exceptions and browser log entries of the attached page, one line each | |
| 84 | +| `network dump` | `Network.*` events written into the same log as lines agent-device's parser reads: method, URL, status, duration, response headers (no bodies) | |
| 85 | +| `perf memory` | `Performance.getMetrics`: JS heap used and total, DOM nodes, documents, listeners | |
| 86 | +| `snapshot`, `find`, `is`, `get`, `wait` | DOM walk over `Runtime.evaluate`: visible interactive, labelled or text nodes, focus from `document.activeElement` | |
| 87 | +| `screenshot` | `Page.captureScreenshot` | |
| 88 | +| `click`, `press <x> <y>`, `longpress`, `hover` | `Input.dispatchMouseEvent` (Magic Remote pointer), ending each move with a small wiggle that wakes the cursor | |
| 89 | +| `fill` | pointer press on the field, select its contents, then `Input.insertText` | |
| 90 | +| `type` | `Input.insertText` into the focused element | |
| 91 | +| `scroll` | `mouseWheel` at the centre of the scroll container or viewport, by the requested distance | |
| 92 | + |
| 93 | +Not yet: `perf frames`, heap snapshots, CPU profiles, recording, URLs and deep links in `open`, |
| 94 | +`media-play-pause` (the LG remote has separate Play and Pause buttons), `power`. |
| 95 | + |
| 96 | +## Known limitations |
| 97 | + |
| 98 | +- **Idle leases expire after 60 s by default, and expiry ends the session.** agent-device gives a |
| 99 | + direct-device provider lease its default TTL and beats it only while a command runs. After a |
| 100 | + longer pause, the next command fails with `Lease is not active` or `No active session` and |
| 101 | + needs a new `connect` and `open`. To avoid it, run `agent-device daemon stop`, then start the |
| 102 | + daemon with `AGENT_DEVICE_LEASE_TTL_MS` and `AGENT_DEVICE_LEASE_MAX_TTL_MS` set (for example |
| 103 | + 30 min). The page itself stays attached across leases. |
| 104 | +- **`network dump` shares the log with console output.** A console line that contains a URL can |
| 105 | + appear as a request. A redirect appears as one line per hop. |
| 106 | +- **`screenshot` captures the app page, not the TV screen.** After `home` the page is hidden and |
| 107 | + renders no frames, so `screenshot` fails at once and names the foreground app; `open <appId>` |
| 108 | + brings the app back. A whole-screen capture is not reachable on a retail TV: SSAP |
| 109 | + `tv/executeOneShot` answers `401 insufficient permissions` to unsigned clients, LG rejects the |
| 110 | + public signed manifest (`blacklisted certificate`), and the dev-mode shell (`prisoner`) gets no |
| 111 | + answer from `luna-send-pub`. |
| 112 | +- **`perf memory` text output says `missing memory metric`.** agent-device's summary reads only |
| 113 | + process memory (RSS/PSS), which CDP does not give without a tracing dump. The JS heap numbers |
| 114 | + are in `--json`. |
| 115 | +- **Console messages from before the attach are replayed without object previews**, so objects |
| 116 | + in them show as `Object`. Messages logged after the attach show their properties. |
| 117 | +- **`open` on a running app can reload it** (webOS relaunch), so the first read may hit a document |
| 118 | + without a body. `open` waits for a loaded document, and a snapshot of an unloaded page is empty |
| 119 | + rather than an error. `open --relaunch` closes the app first, for a cold start. |
| 120 | +- **A second DevTools client on the same page hung in testing.** Close Chrome DevTools and other |
| 121 | + inspector clients attached to the app before a session. |
| 122 | +- **Snapshots are DOM-derived.** Labels come from `aria-label`, `alt`, `title` or own text; |
| 123 | + rects are CSS px, which match screenshot pixels only at `devicePixelRatio` 1 (1080p apps). |
| 124 | + The walk does not enter iframes or shadow roots, so overlays rendered there (for example a |
| 125 | + dev-server error overlay) appear in screenshots but not in snapshots. |
| 126 | +- **System overlays are outside the page.** The LG menu, on-screen keyboard and other system |
| 127 | + layers never appear in `screenshot` or `snapshot`. |
0 commit comments