|
| 1 | +# @agent-device/webos |
| 2 | + |
| 3 | +Drive web apps on LG webOS TVs with [agent-device](https://agent-device.dev). The plugin launches |
| 4 | +the app with the webOS CLI, opens the `ares-inspect` tunnel, and drives the page over the Chrome |
| 5 | +DevTools Protocol: remote keys, pointer, text, snapshots, screenshots and console logs. |
| 6 | + |
| 7 | +Status: experimental. Verified on an LG 32LQ63006LA (webOS TV 22, dev build of a React Native Web |
| 8 | +app): `connect`, `open`, `snapshot -i`, `snapshot -s`, `tv-remote` (directions, select, back), |
| 9 | +`is focused`, `press @ref --settle`, `wait text`, `find`, `screenshot`, `logs`, `apps`, |
| 10 | +`appstate`, `close`, `disconnect`. |
| 11 | + |
| 12 | +## How it plugs in |
| 13 | + |
| 14 | +agent-device leases only iOS, Android and HarmonyOS devices, so a webOS TV enters as an |
| 15 | +**Android TV device that this plugin owns** (`platform: android`, `target: tv`, id |
| 16 | +`webos:<ares device>`). Every command on that device goes to this plugin's runtime, never to adb. |
| 17 | + |
| 18 | +## Requirements |
| 19 | + |
| 20 | +- webOS CLI (`ares-*`) on `PATH`, or the directory that holds it passed as `aresBin`; see LG's |
| 21 | + [webOS CLI installation](https://webostv.developer.lge.com/develop/tools/cli-installation) |
| 22 | +- The TV in Developer Mode and registered with `ares-setup-device`; see LG's |
| 23 | + [App Testing with Developer Mode App](https://webostv.developer.lge.com/develop/getting-started/developer-mode-app) |
| 24 | +- The app installed on the TV |
| 25 | +- The TV and this host on the same network |
| 26 | + |
| 27 | +## Install |
| 28 | + |
| 29 | +```sh |
| 30 | +agent-device plugins add @agent-device/webos |
| 31 | +agent-device daemon stop # the next command starts a daemon with the plugin loaded |
| 32 | +``` |
| 33 | + |
| 34 | +Plugin options: `aresBin` (directory with the `ares-*` CLIs) and `device` (default ares device |
| 35 | +name). |
| 36 | + |
| 37 | +To try a local build, pack the package and install the tarball into a plugin store installation: |
| 38 | + |
| 39 | +```sh |
| 40 | +pnpm --filter @agent-device/webos pack --out /tmp/agent-device-webos.tgz |
| 41 | +installation=$(uuidgen) |
| 42 | +npm install --prefix ~/.agent-device/plugins/$installation /tmp/agent-device-webos.tgz |
| 43 | +``` |
| 44 | + |
| 45 | +Then register `{"plugins": {"@agent-device/webos": {"installation": "<installation>"}}}` in |
| 46 | +`~/.agent-device/config.json` and run `agent-device daemon stop`. |
| 47 | + |
| 48 | +## Use |
| 49 | + |
| 50 | +```sh |
| 51 | +agent-device daemon stop # the lease TTL is read when the daemon starts |
| 52 | +export AGENT_DEVICE_LEASE_TTL_MS=1800000 AGENT_DEVICE_LEASE_MAX_TTL_MS=1800000 # see Known limitations |
| 53 | +agent-device connect webos [--device <ares device>] # defaults to the ares default device |
| 54 | +agent-device open <appId> # ares-launch, then attach over CDP |
| 55 | +agent-device tv-remote press down |
| 56 | +agent-device snapshot -i |
| 57 | +agent-device screenshot ./tv.png |
| 58 | +agent-device logs start # then logs path |
| 59 | +agent-device close <appId> |
| 60 | +agent-device disconnect |
| 61 | +``` |
| 62 | + |
| 63 | +Later commands use the session `connect` created; `connect` prints its name for `--session`. |
| 64 | + |
| 65 | +## Supported |
| 66 | + |
| 67 | +| Command | How | |
| 68 | +| --- | --- | |
| 69 | +| `open` / `close` | `ares-launch` / `ares-launch --close`, then the `ares-inspect` tunnel | |
| 70 | +| `tv-remote press up/down/left/right/select/back`, `back` | `Input.dispatchKeyEvent` with the LG remote codes (Back = 461) | |
| 71 | +| `appstate` | the attached app while its page is visible | |
| 72 | +| `apps` | `ares-install --listfull` | |
| 73 | +| `logs` | console calls, exceptions and browser log entries of the attached page, one line each | |
| 74 | +| `snapshot`, `find`, `is`, `get`, `wait` | DOM walk over `Runtime.evaluate`: visible interactive, labelled or text nodes, focus from `document.activeElement`; `-s`, `-d` and `-i` apply to the walk | |
| 75 | +| `screenshot` | `Page.captureScreenshot` | |
| 76 | +| `click`, `press <x> <y>`, `longpress`, `hover` | `Input.dispatchMouseEvent` (Magic Remote pointer), ending each move with a small wiggle that wakes the cursor | |
| 77 | +| `fill` | pointer press on the field, select its contents, then `Input.insertText`, or Backspace for an empty value | |
| 78 | +| `type` | `Input.insertText` into the focused element | |
| 79 | +| `scroll` | `mouseWheel` at the centre of the scroll container or viewport, by the requested distance | |
| 80 | + |
| 81 | +Not yet: system buttons (`home`, menu, volume, channel and media keys), `network dump`, `perf`, |
| 82 | +`install` and `uninstall`, recording, URLs and deep links in `open`, `power`. |
| 83 | + |
| 84 | +## Known limitations |
| 85 | + |
| 86 | +- **Idle leases expire after 60 s by default, and expiry ends the session.** agent-device gives a |
| 87 | + direct-device provider lease its default TTL and beats it only while a command runs. When the |
| 88 | + lease ends, by `disconnect` or by expiry, the plugin closes the page connection and the |
| 89 | + `ares-inspect` tunnel. After a longer pause, the next command fails with `Lease is not active` |
| 90 | + or `No active session` and needs a new `connect` and `open`. To avoid it, run |
| 91 | + `agent-device daemon stop`, export `AGENT_DEVICE_LEASE_TTL_MS` and |
| 92 | + `AGENT_DEVICE_LEASE_MAX_TTL_MS` (for example `1800000`, 30 min), and run the next command in |
| 93 | + that shell. That command starts the daemon with both values. |
| 94 | +- **`screenshot` captures the app page, not the TV screen.** When the app is in the background, |
| 95 | + for example after Home on the remote, the page is hidden and renders no frames, so `screenshot` |
| 96 | + fails at once; `open <appId>` brings the app back. A whole-screen capture is not reachable on a retail TV: SSAP |
| 97 | + `tv/executeOneShot` answers `401 insufficient permissions` to unsigned clients, LG rejects the |
| 98 | + public signed manifest (`blacklisted certificate`), and the dev-mode shell (`prisoner`) gets no |
| 99 | + answer from `luna-send-pub`. |
| 100 | +- **Console messages from before the attach are replayed without object previews**, so objects |
| 101 | + in them show as `Object`. Messages logged after the attach show their properties. |
| 102 | +- **`open` on a running app can reload it** (webOS relaunch), so the first read may hit a document |
| 103 | + without a body. `open` waits for a loaded document, and a snapshot of an unloaded page is empty |
| 104 | + rather than an error. `open --relaunch` closes the app first, for a cold start. |
| 105 | +- **A second DevTools client on the same page hung in testing.** Close Chrome DevTools and other |
| 106 | + inspector clients attached to the app before a session. |
| 107 | +- **Snapshots are DOM-derived.** Labels come from `aria-label`, `alt`, `title` or own text; |
| 108 | + rects are CSS px, which match screenshot pixels only at `devicePixelRatio` 1 (1080p apps). |
| 109 | + The walk does not enter iframes or shadow roots, so overlays rendered there (for example a |
| 110 | + dev-server error overlay) appear in screenshots but not in snapshots. |
| 111 | +- **System overlays are outside the page.** The LG menu, on-screen keyboard and other system |
| 112 | + layers never appear in `screenshot` or `snapshot`. |
0 commit comments