Skip to content

Commit 1cd5272

Browse files
committed
feat(provider-webos): add LG webOS TV provider plugin
Add @agent-device/webos, an optional provider plugin that drives web apps on LG webOS TVs. The TV enters as an Android TV device owned by the plugin (id webos:<ares device>). Lifecycle runs through ares-launch and ares-install, the page is driven over the Chrome DevTools Protocol through the ares-inspect tunnel, and system buttons and appstate go over SSAP using undici's WebSocket with a self-signed-certificate agent. - Kernel: add the cdp-dom snapshot producer on the android channel so DOM snapshots do not pose as UIAutomator output. - Capability matrix: add an LG webOS TV column that reads its device from the plugin's own lease and inventory; split the hover and viewport row. - Packed-install smoke: let scripts/check-provider-plugin.mjs run fixtures for providers without credentials, and add the webOS fixture. - Docs: add the LG webOS page, mention it in plugins, commands and the sidebar, and add a webOS flow to help remote.
1 parent 3c6673a commit 1cd5272

42 files changed

Lines changed: 2412 additions & 84 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.fallowrc.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
"packages/provider-testmu/test/package-smoke.mjs",
88
"packages/provider-testmu/tsdown.config.ts",
99
"packages/provider-testmu/src/plugin.ts",
10+
"packages/provider-webos/test/package-smoke.mjs",
11+
"packages/provider-webos/tsdown.config.ts",
1012
"vitest.mutation.config.ts",
1113
"src/sdk/index.ts",
1214
"src/sdk/io.ts",

‎examples/sdk/tsconfig.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@
5454
{ "path": "../../packages/plugins" },
5555
{ "path": "../../packages/provider-limrun" },
5656
{ "path": "../../packages/provider-testmu" },
57+
{ "path": "../../packages/provider-webos" },
5758
{ "path": "../../packages/provider-webdriver" },
5859
{ "path": "../../packages/provision-kit" },
5960
{ "path": "../../packages/proxy" },

‎package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,7 @@
366366
"vitest": "5.0.4",
367367
"yaml": "^2.9.0",
368368
"yauzl": "^3.4.0",
369-
"@agent-device/testmu": "workspace:*"
369+
"@agent-device/testmu": "workspace:*",
370+
"@agent-device/webos": "workspace:*"
370371
}
371372
}

‎packages/kernel/src/snapshot.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -586,7 +586,7 @@ export type SnapshotProvenance =
586586
backend: 'xctest';
587587
producer: 'apple-runner' | 'simulator-ax-bridge' | 'appium-source' | 'limrun-ios-tree';
588588
}
589-
| { backend: 'android'; producer: 'android-uiautomator' | 'appium-source' }
589+
| { backend: 'android'; producer: 'android-uiautomator' | 'appium-source' | 'cdp-dom' }
590590
| { backend: 'harmonyos-arkui'; producer: 'harmonyos-uitest' }
591591
| { backend: 'macos-helper'; producer: 'macos-helper' }
592592
| { backend: 'linux-atspi'; producer: 'linux-atspi' }

‎packages/provider-testmu/test/package-smoke.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
export const provider = 'testmu';
12
export const args = [
23
'--platform',
34
'android',

‎packages/provider-webos/README.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# @agent-device/webos
2+
3+
Drive web apps on LG webOS TVs with [agent-device](https://agent-device.dev). The plugin launches
4+
the app with the webOS CLI, opens the `ares-inspect` tunnel, and drives the page over the Chrome
5+
DevTools Protocol: remote keys, pointer, text, snapshots, screenshots, console and network. System
6+
buttons (Home, volume) go to the TV as real remote presses over SSAP, LG's remote protocol.
7+
8+
Status: experimental. Verified on an LG TV (webOS, dev build of a React Native Web app):
9+
`connect`, `open`, `snapshot -i`, `tv-remote` (directions, select, back, volume, home), `home`,
10+
`is focused`, `press @ref --settle`, `wait text`, `find`, `screenshot`, `logs`, `network dump`,
11+
`perf memory --json`, `apps`, `appstate`, `install`, `uninstall`, `close`.
12+
13+
## How it plugs in
14+
15+
agent-device leases only iOS, Android and HarmonyOS devices, so a webOS TV enters as an
16+
**Android TV device that this plugin owns** (`platform: android`, `target: tv`, id
17+
`webos:<ares device>`). Every command on that device goes to this plugin's runtime, never to adb.
18+
19+
## Requirements
20+
21+
- webOS CLI (`ares-*`) on `PATH`, or the directory that holds it passed as `aresBin`; see LG's
22+
[webOS CLI installation](https://webostv.developer.lge.com/develop/tools/cli-installation)
23+
- The TV in Developer Mode and registered with `ares-setup-device`; see LG's
24+
[App Testing with Developer Mode App](https://webostv.developer.lge.com/develop/getting-started/developer-mode-app)
25+
- The app installed on the TV
26+
- The TV and this host on the same network
27+
- For system buttons and `appstate`: SSAP pairing (see below)
28+
29+
## Install
30+
31+
```sh
32+
agent-device plugins add @agent-device/webos
33+
agent-device daemon stop # the next command starts a daemon with the plugin loaded
34+
```
35+
36+
Plugin options: `aresBin` (directory with the `ares-*` CLIs), `device` (default ares device name),
37+
`ssapKeyFile` (where SSAP pairing keys are kept, default `webos-ssap-keys.json` in the
38+
agent-device home).
39+
40+
To try a local build, pack the package and install the tarball into a plugin store installation:
41+
42+
```sh
43+
pnpm --filter @agent-device/webos pack --out /tmp/agent-device-webos.tgz
44+
installation=$(uuidgen)
45+
npm install --prefix ~/.agent-device/plugins/$installation /tmp/agent-device-webos.tgz
46+
```
47+
48+
Then register `{"plugins": {"@agent-device/webos": {"installation": "<installation>"}}}` in
49+
`~/.agent-device/config.json` and run `agent-device daemon stop`.
50+
51+
### SSAP pairing
52+
53+
The first system-button press or `appstate` on a TV shows a pairing prompt on screen. Someone at
54+
the TV accepts it with the remote within 60 s. The client key is then stored per TV IP in
55+
`ssapKeyFile`, and later presses connect without a prompt. A new IP prompts again.
56+
57+
## Use
58+
59+
```sh
60+
agent-device daemon stop # the lease TTL is read when the daemon starts
61+
export AGENT_DEVICE_LEASE_TTL_MS=1800000 AGENT_DEVICE_LEASE_MAX_TTL_MS=1800000 # see Known limitations
62+
agent-device connect webos [--device <ares device>] # defaults to the ares default device
63+
agent-device open <appId> # ares-launch, then attach over CDP
64+
agent-device tv-remote press down
65+
agent-device snapshot -i
66+
agent-device screenshot ./tv.png
67+
agent-device logs start # then logs path / network dump
68+
agent-device close <appId>
69+
agent-device disconnect
70+
```
71+
72+
Later commands use the session `connect` created; `connect` prints its name for `--session`.
73+
74+
## Supported
75+
76+
| Command | How |
77+
| --- | --- |
78+
| `open` / `close` | `ares-launch` / `ares-launch --close`, then the `ares-inspect` tunnel |
79+
| `tv-remote press up/down/left/right/select/back`, `back` | `Input.dispatchKeyEvent` with the LG remote codes (Back = 461) |
80+
| `tv-remote press home/menu/volume-up/volume-down/mute/channel-up/channel-down/media-rewind/media-fast-forward`, `home` | SSAP pointer-socket button presses |
81+
| `appstate` | SSAP `getForegroundAppInfo` |
82+
| `apps`, `install <appId> <ipk>`, `uninstall <appId>` | `ares-install --listfull` / `ares-install` / `ares-install --remove` |
83+
| `logs` | console calls, exceptions and browser log entries of the attached page, one line each |
84+
| `network dump` | `Network.*` events written into the same log as lines agent-device's parser reads: method, URL, status, duration, response headers (no bodies) |
85+
| `perf memory` | `Performance.getMetrics`: JS heap used and total, DOM nodes, documents, listeners |
86+
| `snapshot`, `find`, `is`, `get`, `wait` | DOM walk over `Runtime.evaluate`: visible interactive, labelled or text nodes, focus from `document.activeElement` |
87+
| `screenshot` | `Page.captureScreenshot` |
88+
| `click`, `press <x> <y>`, `longpress`, `hover` | `Input.dispatchMouseEvent` (Magic Remote pointer), ending each move with a small wiggle that wakes the cursor |
89+
| `fill` | pointer press on the field, select its contents, then `Input.insertText` |
90+
| `type` | `Input.insertText` into the focused element |
91+
| `scroll` | `mouseWheel` at the centre of the scroll container or viewport, by the requested distance |
92+
93+
Not yet: `perf frames`, heap snapshots, CPU profiles, recording, URLs and deep links in `open`,
94+
`media-play-pause` (the LG remote has separate Play and Pause buttons), `power`.
95+
96+
## Known limitations
97+
98+
- **Idle leases expire after 60 s by default, and expiry ends the session.** agent-device gives a
99+
direct-device provider lease its default TTL and beats it only while a command runs. After a
100+
longer pause, the next command fails with `Lease is not active` or `No active session` and
101+
needs a new `connect` and `open`. To avoid it, run `agent-device daemon stop`, then start the
102+
daemon with `AGENT_DEVICE_LEASE_TTL_MS` and `AGENT_DEVICE_LEASE_MAX_TTL_MS` set (for example
103+
30 min). The page itself stays attached across leases.
104+
- **`network dump` shares the log with console output.** A console line that contains a URL can
105+
appear as a request. A redirect appears as one line per hop.
106+
- **`screenshot` captures the app page, not the TV screen.** After `home` the page is hidden and
107+
renders no frames, so `screenshot` fails at once and names the foreground app; `open <appId>`
108+
brings the app back. A whole-screen capture is not reachable on a retail TV: SSAP
109+
`tv/executeOneShot` answers `401 insufficient permissions` to unsigned clients, LG rejects the
110+
public signed manifest (`blacklisted certificate`), and the dev-mode shell (`prisoner`) gets no
111+
answer from `luna-send-pub`.
112+
- **`perf memory` text output says `missing memory metric`.** agent-device's summary reads only
113+
process memory (RSS/PSS), which CDP does not give without a tracing dump. The JS heap numbers
114+
are in `--json`.
115+
- **Console messages from before the attach are replayed without object previews**, so objects
116+
in them show as `Object`. Messages logged after the attach show their properties.
117+
- **`open` on a running app can reload it** (webOS relaunch), so the first read may hit a document
118+
without a body. `open` waits for a loaded document, and a snapshot of an unloaded page is empty
119+
rather than an error. `open --relaunch` closes the app first, for a cold start.
120+
- **A second DevTools client on the same page hung in testing.** Close Chrome DevTools and other
121+
inspector clients attached to the app before a session.
122+
- **Snapshots are DOM-derived.** Labels come from `aria-label`, `alt`, `title` or own text;
123+
rects are CSS px, which match screenshot pixels only at `devicePixelRatio` 1 (1080p apps).
124+
The walk does not enter iframes or shadow roots, so overlays rendered there (for example a
125+
dev-server error overlay) appear in screenshots but not in snapshots.
126+
- **System overlays are outside the page.** The LG menu, on-screen keyboard and other system
127+
layers never appear in `screenshot` or `snapshot`.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"name": "@agent-device/webos",
3+
"version": "0.21.36-dev",
4+
"type": "module",
5+
"description": "LG webOS TV web app plugin for agent-device.",
6+
"files": [
7+
"dist"
8+
],
9+
"scripts": {
10+
"build": "tsdown --config tsdown.config.ts",
11+
"prepack": "pnpm build"
12+
},
13+
"exports": {
14+
".": {
15+
"types": "./src/index.ts",
16+
"default": "./src/index.ts"
17+
},
18+
"./agent-device": {
19+
"types": "./src/index.ts",
20+
"default": "./src/index.ts"
21+
},
22+
"./package.json": "./package.json"
23+
},
24+
"devDependencies": {
25+
"@agent-device/capture-kit": "workspace:*",
26+
"@agent-device/contracts": "workspace:*",
27+
"@agent-device/host-kit": "workspace:*",
28+
"@agent-device/kernel": "workspace:*",
29+
"@agent-device/test-kit": "workspace:*",
30+
"agent-device": "workspace:*",
31+
"undici": "7.30.1"
32+
},
33+
"publishConfig": {
34+
"exports": {
35+
".": {
36+
"import": "./dist/agent-device.mjs"
37+
},
38+
"./agent-device": {
39+
"import": "./dist/agent-device.mjs"
40+
}
41+
},
42+
"access": "public"
43+
},
44+
"license": "MIT",
45+
"repository": {
46+
"type": "git",
47+
"url": "git+https://github.com/callstack/agent-device.git",
48+
"directory": "packages/provider-webos"
49+
},
50+
"homepage": "https://agent-device.dev/docs/webos"
51+
}
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
import assert from 'node:assert/strict';
2+
import { chmodSync, writeFileSync } from 'node:fs';
3+
import path from 'node:path';
4+
import { mkdtempForTestSync } from '@agent-device/test-kit';
5+
import { test } from 'vitest';
6+
7+
import { createAres } from './ares.ts';
8+
9+
function fakeAresBin(outputs: Record<string, string>) {
10+
const bin = mkdtempForTestSync('ares-');
11+
for (const [name, output] of Object.entries(outputs)) {
12+
const script = path.join(bin, name);
13+
writeFileSync(script, `#!/bin/sh\ncat <<'OUTPUT'\n${output}\nOUTPUT\n`);
14+
chmodSync(script, 0o755);
15+
}
16+
return bin;
17+
}
18+
19+
test('listDevices returns only TV profiles with their IP and default flag', async () => {
20+
const bin = fakeAresBin({
21+
'ares-setup-device': JSON.stringify([
22+
{ profile: 'tv', name: 'living-room', default: true, deviceinfo: { ip: '192.0.2.10' } },
23+
{ profile: 'ose', name: 'signage', default: false, deviceinfo: { ip: '192.0.2.20' } },
24+
]),
25+
});
26+
27+
const devices = await createAres(bin, process.env).listDevices();
28+
29+
assert.deepEqual(devices, [{ name: 'living-room', ip: '192.0.2.10', isDefault: true }]);
30+
});
31+
32+
test('listApps reads each app id with its title from ares-install --listfull', async () => {
33+
const bin = fakeAresBin({
34+
'ares-install': [
35+
'[Info] Set target device : living-room',
36+
'id : com.sdk.ares.hostedapp',
37+
'title : Ares Hosted App',
38+
'version : 1.1.0',
39+
'',
40+
'id : com.example.tvapp',
41+
'title : Example TV',
42+
'requiredACG',
43+
'',
44+
'id : com.example.untitled',
45+
].join('\n'),
46+
});
47+
48+
const apps = await createAres(bin, process.env).forDevice('living-room').listApps();
49+
50+
assert.deepEqual(apps, [
51+
{ id: 'com.sdk.ares.hostedapp', name: 'Ares Hosted App' },
52+
{ id: 'com.example.tvapp', name: 'Example TV' },
53+
{ id: 'com.example.untitled', name: 'com.example.untitled' },
54+
]);
55+
});
56+
57+
test('a missing webOS CLI fails as TOOL_MISSING with a hint naming aresBin', async () => {
58+
const bin = fakeAresBin({});
59+
60+
const listing = createAres(bin, process.env).listDevices();
61+
62+
await assert.rejects(listing, (error: { code?: string; details?: { hint?: string } }) => {
63+
assert.equal(error.code, 'TOOL_MISSING');
64+
assert.match(error.details?.hint ?? '', /aresBin/);
65+
return true;
66+
});
67+
});

0 commit comments

Comments
 (0)