Skip to content

Commit fb6a58a

Browse files
thymikeethymikee
committed
feat(android): detect Fire OS devices as Android TV targets and document Fire TV support
Fire TV devices running Fire OS are Android-based and automate over ADB, but TV detection only read the AOSP leanback/television features and app launch used the LEANBACK_LAUNCHER category alone. Fire OS devices declare amazon.hardware.fire_tv and may declare no leanback feature, and Fire OS apps can register under either launcher category. - Inventory: treat amazon.hardware.fire_tv as a TV feature signal. - App launch and listing on a TV target: query LEANBACK_LAUNCHER first, fall back to LAUNCHER, and launch the resolved component under the category it resolved under. - Docs and versioned help: Fire TV (Fire OS) connects as an Android TV device; Vega OS remains VVD-only and is a different OS; Electron/Chromium macOS apps automate through the accessibility tree.
1 parent b08b5e7 commit fb6a58a

17 files changed

Lines changed: 207 additions & 30 deletions

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717

1818
<!-- Intro rule: category line, job pitch with the platform list, works-with/proof lines, nothing else. Per-platform transports, tool names, vendor lists, and support caveats belong in "How it works" and website/docs, never here. Every name in the proof lines must link to public evidence. -->
1919

20-
Let your coding agent verify its changes in the running app. `agent-device` lets agents inspect, control, debug, and verify apps on iOS, Android, and HarmonyOS (simulators, emulators, and physical devices), plus tvOS, Android TV, Amazon Vega OS TV (Vega Virtual Device), web, macOS, and Linux. Agents read token-efficient accessibility snapshots instead of reasoning over screenshots alone, act through refs and selectors, and save evidence for review. It also coordinates device access across parallel agent worktrees and connects to remote device clouds.
20+
Let your coding agent verify its changes in the running app. `agent-device` lets agents inspect, control, debug, and verify apps on iOS, Android, and HarmonyOS (simulators, emulators, and physical devices), plus tvOS, Android TV, Fire TV (Fire OS), Amazon Vega OS TV (Vega Virtual Device), web, macOS, and Linux. Agents read token-efficient accessibility snapshots instead of reasoning over screenshots alone, act through refs and selectors, and save evidence for review. It also coordinates device access across parallel agent worktrees and connects to remote device clouds.
2121

2222
Works with Claude Code, Codex, Cursor, Windsurf, Cline, Goose, and any agent that can run a CLI or connect over MCP, or as the runtime under agents you build with the [AI SDK](https://oss.callstack.com/agent-device/docs/ai-sdk) or [Eve](https://oss.callstack.com/agent-device/docs/eve). Developers at [Expensify](https://www.callstack.com/blog/how-expensify-uses-agent-device-for-mobile-bug-evidence-and-profiling), [Shopify](https://x.com/mustafa01ali/status/2036577353178943826), and [others](#who-uses-agent-device) use it to verify their apps.
2323

@@ -155,7 +155,7 @@ The same session and evidence model works at every step: the agent explores the
155155

156156
## How it works
157157

158-
`agent-device` keeps device state in sessions. It uses a local accessibility bridge for iOS Simulator snapshots and XCTest for iOS interactions, physical iOS, and tvOS; ADB and the snapshot helper on Android; HDC and ArkUI `uitest` on HarmonyOS; Vega CLI/VDA on the Vega Virtual Device; a local helper on macOS; and AT-SPI on Linux.
158+
`agent-device` keeps device state in sessions. It uses a local accessibility bridge for iOS Simulator snapshots and XCTest for iOS interactions, physical iOS, and tvOS; ADB and the snapshot helper on Android, Android TV, and Fire OS; HDC and ArkUI `uitest` on HarmonyOS; Vega CLI/VDA on the Vega Virtual Device; a local helper on macOS, including Electron and other Chromium-based apps through the accessibility tree; and AT-SPI on Linux.
159159

160160
Support depth varies by target. Some backends, such as HarmonyOS and Vega OS, cover a subset of commands; run `agent-device capabilities --platform <platform>` to see what a target supports.
161161

‎packages/platform-android/src/__tests__/app-helpers.test.ts‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,32 @@ async function withMockedAdbScript(script: string, run: () => Promise<void>): Pr
2424
}
2525
}
2626

27+
test('listAndroidAppsWithAdb queries both launcher categories for a TV target', async () => {
28+
// Fire OS apps register under either launcher category, so a TV listing merges both queries.
29+
const calls: (readonly string[])[] = [];
30+
const adb: AndroidAdbExecutor = async (args) => {
31+
calls.push(args);
32+
if (args.includes('query-activities')) {
33+
return {
34+
exitCode: 0,
35+
stdout: args.includes('android.intent.category.LAUNCHER')
36+
? 'com.example.fireosapp/.MainActivity\n'
37+
: '',
38+
stderr: '',
39+
};
40+
}
41+
return { exitCode: 0, stdout: '', stderr: '' };
42+
};
43+
44+
const apps = await listAndroidAppsWithAdb(adb, { filter: 'all', target: 'tv' });
45+
46+
assert.deepEqual(apps, [{ package: 'com.example.fireosapp', name: 'Fireosapp' }]);
47+
assert.deepEqual(calls.map((call) => call.at(-1)).sort(), [
48+
'android.intent.category.LAUNCHER',
49+
'android.intent.category.LEANBACK_LAUNCHER',
50+
]);
51+
});
52+
2753
test('listAndroidAppsWithAdb uses an injected executor', async () => {
2854
const calls: (readonly string[])[] = [];
2955
const adb: AndroidAdbExecutor = async (args) => {

‎packages/platform-android/src/__tests__/app-lifecycle-open.test.ts‎

Lines changed: 117 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { test } from 'vitest';
22
import assert from 'node:assert/strict';
3-
import { openAndroidApp } from '../app-lifecycle.ts';
3+
import { listAndroidApps, openAndroidApp } from '../app-lifecycle.ts';
44
import { withAndroidAdbProvider } from '../adb-executor.ts';
55
import type { DeviceInfo } from '@agent-device/kernel/device';
66
import { AppError } from '@agent-device/kernel/errors';
@@ -755,6 +755,122 @@ test('openAndroidApp fallback resolve-activity includes MAIN/LAUNCHER flags', as
755755
);
756756
});
757757

758+
test('openAndroidApp TV fallback carries the resolved LAUNCHER category into the component launch', async () => {
759+
// A Fire OS app that declares no LEANBACK_LAUNCHER activity resolves under LAUNCHER only; the
760+
// fallback launch must carry that category, not the target's LEANBACK_LAUNCHER.
761+
await withFakeAdb(
762+
(args) => {
763+
if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'start') {
764+
if (args.includes('-p')) {
765+
return [
766+
'Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.DEFAULT,android.intent.category.LEANBACK_LAUNCHER] pkg=com.example.tvapp }',
767+
'Error: Activity not started, unable to resolve Intent { act=android.intent.action.MAIN cat=[android.intent.category.DEFAULT,android.intent.category.LEANBACK_LAUNCHER] flg=0x10000000 pkg=com.example.tvapp }',
768+
].join('\n');
769+
}
770+
return 'Status: ok';
771+
}
772+
if (
773+
args[0] === 'shell' &&
774+
args[1] === 'cmd' &&
775+
args[2] === 'package' &&
776+
args[3] === 'resolve-activity'
777+
) {
778+
if (args.includes('android.intent.category.LEANBACK_LAUNCHER')) {
779+
return { stdout: '', stderr: 'No activity found', exitCode: 1 };
780+
}
781+
return 'com.example.tvapp/com.example.tvapp.MainActivity';
782+
}
783+
return undefined;
784+
},
785+
async ({ calls, device }) => {
786+
await openAndroidApp(device, 'com.example.tvapp');
787+
assert.deepEqual(calls, [
788+
[
789+
'shell',
790+
'am',
791+
'start',
792+
'-W',
793+
'-a',
794+
'android.intent.action.MAIN',
795+
'-c',
796+
'android.intent.category.DEFAULT',
797+
'-c',
798+
'android.intent.category.LEANBACK_LAUNCHER',
799+
'-p',
800+
'com.example.tvapp',
801+
],
802+
// resolve-activity tries LEANBACK_LAUNCHER first on a TV target...
803+
[
804+
'shell',
805+
'cmd',
806+
'package',
807+
'resolve-activity',
808+
'--brief',
809+
'-a',
810+
'android.intent.action.MAIN',
811+
'-c',
812+
'android.intent.category.LEANBACK_LAUNCHER',
813+
'com.example.tvapp',
814+
],
815+
// ...then LAUNCHER, which resolves the component.
816+
[
817+
'shell',
818+
'cmd',
819+
'package',
820+
'resolve-activity',
821+
'--brief',
822+
'-a',
823+
'android.intent.action.MAIN',
824+
'-c',
825+
'android.intent.category.LAUNCHER',
826+
'com.example.tvapp',
827+
],
828+
// The fallback launch uses the category the component resolved under.
829+
[
830+
'shell',
831+
'am',
832+
'start',
833+
'-W',
834+
'-a',
835+
'android.intent.action.MAIN',
836+
'-c',
837+
'android.intent.category.DEFAULT',
838+
'-c',
839+
'android.intent.category.LAUNCHER',
840+
'-n',
841+
'com.example.tvapp/com.example.tvapp.MainActivity',
842+
],
843+
]);
844+
},
845+
{ device: { ...ANDROID_EMULATOR, target: 'tv' } },
846+
);
847+
});
848+
849+
test('listAndroidApps on a TV target queries LEANBACK_LAUNCHER then LAUNCHER', async () => {
850+
await withFakeAdb(
851+
(args) => {
852+
if (args.includes('query-activities')) {
853+
return args.includes('android.intent.category.LAUNCHER')
854+
? 'com.example.fireosapp/.MainActivity\n'
855+
: '';
856+
}
857+
return undefined;
858+
},
859+
async ({ calls, device }) => {
860+
const apps = await listAndroidApps(device, 'all');
861+
assert.deepEqual(apps, [{ package: 'com.example.fireosapp', name: 'Fireosapp' }]);
862+
const queriedCategories = calls
863+
.filter((call) => call.includes('query-activities'))
864+
.map((call) => call.at(-1));
865+
assert.deepEqual(queriedCategories, [
866+
'android.intent.category.LEANBACK_LAUNCHER',
867+
'android.intent.category.LAUNCHER',
868+
]);
869+
},
870+
{ device: { ...ANDROID_EMULATOR, target: 'tv' } },
871+
);
872+
});
873+
758874
// In-process replacement for the old PATH-stub default adb script: any
759875
// `am start` succeeds with the "Status: ok" marker (so the primary launch
760876
// never falls back to resolve-activity), everything else is empty success.

‎packages/platform-android/src/app-helpers.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,9 @@ function resolveAndroidLaunchCategoriesForAdb(target: AndroidAppListTarget): str
5454
case 'mobile':
5555
return [ANDROID_LAUNCHER_CATEGORY];
5656
case 'tv':
57-
return [ANDROID_LEANBACK_CATEGORY];
57+
// Fire OS TV apps register under either launcher category, so a TV listing also reads
58+
// LAUNCHER activities; discovery merges the two category queries into one set.
59+
return [ANDROID_LEANBACK_CATEGORY, ANDROID_LAUNCHER_CATEGORY];
5860
default:
5961
return [ANDROID_LAUNCHER_CATEGORY, ANDROID_LEANBACK_CATEGORY];
6062
}

‎packages/platform-android/src/app-lifecycle.ts‎

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ export async function listAndroidApps(
5454
async function listAndroidLaunchablePackages(device: DeviceInfo): Promise<Set<string>> {
5555
const packages = new Set<string>();
5656
for (const category of resolveAndroidLaunchCategories(device, {
57-
includeFallbackWhenUnknown: true,
57+
includeLauncherFallback: true,
5858
})) {
5959
const result = await runAndroidShell(
6060
device,
@@ -86,15 +86,19 @@ function resolveAndroidLauncherCategory(device: DeviceInfo): string {
8686

8787
function resolveAndroidLaunchCategories(
8888
device: DeviceInfo,
89-
options: { includeFallbackWhenUnknown?: boolean } = {},
89+
options: { includeLauncherFallback?: boolean } = {},
9090
): string[] {
9191
if (device.target === 'tv') {
92-
return [ANDROID_LEANBACK_CATEGORY];
92+
// Fire OS TV apps register under either launcher category, so a TV launch or listing that can
93+
// fall back also queries LAUNCHER after LEANBACK_LAUNCHER.
94+
return options.includeLauncherFallback
95+
? [ANDROID_LEANBACK_CATEGORY, ANDROID_LAUNCHER_CATEGORY]
96+
: [ANDROID_LEANBACK_CATEGORY];
9397
}
9498
if (device.target === 'mobile') {
9599
return [ANDROID_LAUNCHER_CATEGORY];
96100
}
97-
if (options.includeFallbackWhenUnknown) {
101+
if (options.includeLauncherFallback) {
98102
return [ANDROID_LAUNCHER_CATEGORY, ANDROID_LEANBACK_CATEGORY];
99103
}
100104
return [ANDROID_LAUNCHER_CATEGORY];
@@ -405,16 +409,22 @@ async function openAndroidPackage(
405409
if (primaryResult.exitCode === 0 && !isAmStartError(primaryResult.stdout, primaryResult.stderr)) {
406410
return primaryResult;
407411
}
408-
const component = await resolveAndroidLaunchComponent(device, packageName);
409-
if (!component) {
412+
const resolvedComponent = await resolveAndroidLaunchComponent(device, packageName);
413+
if (!resolvedComponent) {
410414
if (!(await isAndroidPackageInstalled(device, packageName))) {
411415
throw buildAndroidPackageNotInstalledError(packageName);
412416
}
413417
throw androidAdbResultError(`Failed to launch ${packageName}`, primaryResult);
414418
}
419+
// The fallback launches under the category the component actually resolved under, which on a TV
420+
// target can be LAUNCHER for an app that declares no LEANBACK_LAUNCHER activity.
415421
return await runAndroidShell(
416422
device,
417-
buildAndroidActivityLaunchArgs(component, launchCategory, options),
423+
buildAndroidActivityLaunchArgs(
424+
resolvedComponent.component,
425+
resolvedComponent.category,
426+
options,
427+
),
418428
);
419429
}
420430

@@ -516,9 +526,9 @@ function looksLikeMissingAndroidPackageOutput(output: string): boolean {
516526
async function resolveAndroidLaunchComponent(
517527
device: DeviceInfo,
518528
packageName: string,
519-
): Promise<string | null> {
529+
): Promise<{ component: string; category: string } | null> {
520530
const categories = Array.from(
521-
new Set(resolveAndroidLaunchCategories(device, { includeFallbackWhenUnknown: true })),
531+
new Set(resolveAndroidLaunchCategories(device, { includeLauncherFallback: true })),
522532
);
523533
for (const category of categories) {
524534
const result = await runAndroidShell(
@@ -540,7 +550,7 @@ async function resolveAndroidLaunchComponent(
540550
continue;
541551
}
542552
const component = parseAndroidLaunchComponent(result.stdout);
543-
if (component) return component;
553+
if (component) return { component, category };
544554
}
545555
return null;
546556
}

‎packages/platform-android/src/inventory.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,9 @@ test.each([
145145
test.each([
146146
['feature:android.software.leanback_only\nfeature:android.hardware.wifi\n', 'tv'],
147147
['feature:android.hardware.type.television=1\n', 'tv'],
148+
['feature:android.hardware.wifi\nfeature:amazon.hardware.fire_tv\n', 'tv'],
148149
['feature:android.software.leanback_extra\nfeature:android.hardware.wifi\n', 'mobile'],
150+
['feature:android.hardware.type.automotive\nfeature:android.hardware.wifi\n', 'mobile'],
149151
] as const)(
150152
'Android inventory reads the feature list %j as a %s target',
151153
async (featureList, target) => {

‎packages/platform-android/src/inventory.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,14 @@ const DEVICE_DESCRIPTION_TIMEOUT_MS = 2_000;
2929
const DEVICE_DESCRIPTION_PROBE = shellFragment(
3030
'getprop ro.product.model; getprop ro.build.version.release',
3131
);
32+
// Fire OS devices declare amazon.hardware.fire_tv and may not declare the AOSP leanback
33+
// features, so TV detection must read it as a TV signal too (Amazon: "Identify Fire TV
34+
// Devices", adb shell pm list features).
3235
const TV_FEATURES = [
3336
'android.software.leanback',
3437
'android.software.leanback_only',
3538
'android.hardware.type.television',
39+
'amazon.hardware.fire_tv',
3640
] as const;
3741

3842
type AndroidInventoryContext = Readonly<{

‎skills/agent-device/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: agent-device
3-
description: Automates Apple-platform apps (iOS, tvOS, macOS), Android devices, and Amazon Vega OS TV apps in Vega Virtual Devices. Use when navigating apps, taking snapshots/screenshots where supported, driving TV remotes, tapping, typing, scrolling, extracting UI info, collecting evidence, or planning agent-device CLI commands.
3+
description: Automates Apple-platform apps (iOS, tvOS, macOS), Android devices (including Android TV and Fire TV on Fire OS), and Amazon Vega OS TV apps in Vega Virtual Devices. Use when navigating apps, taking snapshots/screenshots where supported, driving TV remotes, tapping, typing, scrolling, extracting UI info, collecting evidence, or planning agent-device CLI commands.
44
---
55

66
# agent-device

‎src/commands/schema/cli-help-first-screen.test.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,9 @@ const FIRST_SCREEN_RULES: Record<string, readonly string[]> = {
5656
'Rare iOS accessibility gap: a row shown disabled/hittable:false where press reports success but no UI change, or a collapsed composite control with no child refs -- run snapshot -i --json, compute the target center from rects, press x y, then diff snapshot -i. Coordinates are fallback-only; document why you used them.',
5757
],
5858
tv: [
59-
'Use this when the target is Android TV, Apple TV/tvOS, or an Amazon Vega OS TV app running in the Vega Virtual Device (VVD). TV surfaces are focus-first: move focus with remote/D-pad buttons, then activate the focused control',
59+
'Use this when the target is Android TV (including Fire TV devices on Fire OS over ADB), Apple TV/tvOS, or an Amazon Vega OS TV app running in the Vega Virtual Device (VVD). TV surfaces are focus-first: move focus with remote/D-pad buttons, then activate the focused control',
6060
'Vega OS is driven through the SDK-matched Vega CLI and VDA, not ADB',
61-
'Initial support is VVD-only. Physical Fire TV devices remain unsupported',
61+
'Initial support is VVD-only. Physical devices that run Vega OS remain unsupported',
6262
'The VVD is never booted implicitly',
6363
'Snapshot, screenshot, selectors, install, touch/text/gesture, logs, and performance commands remain unsupported until their Vega backends are implemented',
6464
],

‎src/commands/schema/cli-help-topics.test.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,8 @@ test('usageForCommand resolves tv help topic', async () => {
359359
assert.match(help, /Vega OS uses the exact hold duration through inputd-cli/);
360360
assert.match(help, /Use --platform vega --target tv/);
361361
assert.match(help, /Initial support is VVD-only/);
362-
assert.match(help, /Physical Fire TV devices remain unsupported/);
362+
assert.match(help, /Physical devices that run Vega OS remain unsupported/);
363+
assert.match(help, /amazon\.hardware\.fire_tv/);
363364
assert.equal(help.match(/\nVega OS:/g)?.length, 1);
364365
});
365366

0 commit comments

Comments
 (0)