Skip to content

Commit 0cb6fd0

Browse files
committed
fix: close the doc-snippet compiler's stubbing hole and the two suppressed gaps
Addresses the second round of review feedback on #1463's drift guard: 1. stubFreeNamesAndRecompile auto-stubbed every "Cannot find name" as `any`, so a typo like `cliet.apps.open()` would silently pass on the second compile. It now only stubs identifiers in an explicit allowlist (KNOWN_FREE_NAME_STUB_TYPES) — the real SDK-derived continuations (`client`, `androidClient`, `snapshot`) plus the doc's own invented host-glue names, each typed precisely rather than loosely. Anything else is left as a real compile failure. Added a regression test that feeds a `cliet` typo through the guard and asserts it fails. 2. KNOWN_DOC_GAPS filtered six real compiler errors out of the final assertion while the test claimed every snippet compiles. Investigated both and fixed the actual contracts instead of suppressing them: - `prepareMetroRuntime`/`reloadMetro` (src/metro/client-metro.ts) and `stopMetroTunnel` (src/metro/metro.ts) already existed and matched the doc's described workflow almost exactly (same result shape) but were never re-exported from `agent-device/metro`; same for `resolveRemoteConfigProfile` and `agent-device/remote-config`. Added the four exports and fixed the doc's stale function names (`prepareRemoteMetro`/`reloadRemoteMetro`) and one stale field name (`profileKey` -> `companionProfileKey` on the prepare call) to match. - `NetworkOptions`/`AudioOptions` (src/contracts/client-observability.ts) had no `platform` field even though the CLI's `network`/`audio` commands accept `--platform` for the same use case, and the client methods already forward the options object to the daemon generically (`executeCommand('network'|'audio', options)`) — so this was a type gap, not a runtime one. Switched both from AgentDeviceRequestOverrides to DeviceCommandBaseOptions (matching PerfOptions' existing pattern), closing the gap for real instead of stripping `platform` from the doc. - The "Android installFromSource()" snippet was missing its `createAgentDeviceClient` import outright; added it. KNOWN_DOC_GAPS is gone — every fenced snippet now compiles for real, and the test's assertion matches what it claims. 3. Switched the raw `execFileSync` calls to `runCmdSync` from src/utils/exec.ts, per AGENTS.md's process-execution invariant (this is a .ts integration test, not a packaging fixture that needs to stay dependency-free). Refs #1463
1 parent 0da436c commit 0cb6fd0

5 files changed

Lines changed: 77 additions & 67 deletions

File tree

‎src/contracts/client-observability.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,13 @@ export type EventsOptions = AgentDeviceRequestOverrides & {
2828
limit?: number;
2929
};
3030

31-
export type NetworkOptions = AgentDeviceRequestOverrides & {
31+
export type NetworkOptions = DeviceCommandBaseOptions & {
3232
action?: 'dump' | 'log';
3333
limit?: number;
3434
include?: NetworkIncludeMode;
3535
};
3636

37-
export type AudioOptions = AgentDeviceRequestOverrides & {
37+
export type AudioOptions = DeviceCommandBaseOptions & {
3838
action?: 'probe';
3939
probeAction?: 'start' | 'status' | 'stop';
4040
durationMs?: number;

‎src/sdk/metro.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,5 @@ export type {
44
MetroTunnelRequestMessage,
55
MetroTunnelResponseMessage,
66
} from '../metro/metro.ts';
7-
export { resolveRuntimeTransport } from '../metro/metro.ts';
7+
export { resolveRuntimeTransport, stopMetroTunnel } from '../metro/metro.ts';
8+
export { prepareMetroRuntime, reloadMetro } from '../metro/client-metro.ts';

‎src/sdk/remote-config.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
export type { RemoteConfigProfile } from '../remote/remote-config-schema.ts';
2+
export { resolveRemoteConfigProfile } from '../remote/remote-config.ts';

‎test/integration/client-api-doc-snippets.test.ts‎

Lines changed: 64 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ import os from 'node:os';
33
import path from 'node:path';
44
import assert from 'node:assert/strict';
55
import test from 'node:test';
6-
import { execFileSync } from 'node:child_process';
76
import { fileURLToPath } from 'node:url';
7+
import { runCmdSync } from '../../src/utils/exec.ts';
88

99
// Compiles every fenced ```ts snippet in website/docs/docs/client-api.md against
1010
// agent-device's real `agent-device/*` subpath sources, so a doc snippet that no
@@ -36,9 +36,7 @@ function extractTsSnippets(markdown: string): string[] {
3636
// returns it fully resolved and comment-free, so this reads the one true copy of
3737
// the subpath map instead of keeping a second, driftable one here.
3838
function resolveExamplesSdkPaths(): Record<string, string[]> {
39-
const raw = execFileSync(TSC_BIN, ['-p', EXAMPLES_SDK_TSCONFIG, '--showConfig'], {
40-
encoding: 'utf8',
41-
});
39+
const raw = runCmdSync(TSC_BIN, ['-p', EXAMPLES_SDK_TSCONFIG, '--showConfig']).stdout;
4240
const configDir = path.dirname(EXAMPLES_SDK_TSCONFIG);
4341
const paths: Record<string, string[]> = {};
4442
for (const [specifier, targets] of Object.entries(
@@ -49,39 +47,31 @@ function resolveExamplesSdkPaths(): Record<string, string[]> {
4947
return paths;
5048
}
5149

52-
// Free identifiers a snippet references without declaring — continuing a
53-
// `client`/`snapshot` from an earlier snippet in the doc's prose. Typed against
54-
// the real SDK return types (not `any`) so continuation snippets still get
55-
// meaningful checking; anything else (illustrative host-glue names the doc
56-
// invents, like a bridge's own transport function) falls back to `any`, which
57-
// only weakens the check for that invented name, not for the real SDK calls
58-
// around it.
59-
const KNOWN_CONTINUATION_STUB_TYPES: Record<string, string> = {
50+
// Free identifiers a snippet references without declaring, because the doc's
51+
// prose treats them as continuing from an earlier snippet (`client`,
52+
// `androidClient`, `snapshot` — typed against the real SDK return type, not
53+
// `any`, so continuation snippets still get meaningful checking) or as
54+
// illustrative host-glue the doc invents on purpose (a bridge's own transport
55+
// function, never part of the SDK). This is a closed, explicit allowlist, not
56+
// a catch-all: any free identifier NOT listed here — including a typo of one
57+
// that IS, like `cliet` for `client` — is left as a real "Cannot find name"
58+
// failure. See the "rejects an unrecognized free identifier" test below for
59+
// the regression this guards against.
60+
const KNOWN_FREE_NAME_STUB_TYPES: Record<string, string> = {
6061
client: `ReturnType<typeof import('agent-device').createAgentDeviceClient>`,
6162
androidClient: `ReturnType<typeof import('agent-device').createAgentDeviceClient>`,
6263
snapshot: `Awaited<ReturnType<ReturnType<typeof import('agent-device').createAgentDeviceClient>['capture']['snapshot']>>`,
64+
// "Android ADB providers": the doc's own invented remote-transport glue, not
65+
// part of agent-device — typed as the real `AndroidAdbExecutor` function
66+
// shape so the snippet's `exec: async (args, options) => ...` still has to
67+
// return something assignable to it.
68+
runAdbThroughRemoteTunnel: `import('agent-device/android-adb').AndroidAdbExecutor`,
69+
// "Batch orchestration for custom transports": the doc's own invented
70+
// command dispatcher and error mapper.
71+
dispatch: `(stepReq: unknown) => Promise<import('agent-device/contracts').DaemonResponseData>`,
72+
bridgeErrorToDaemonResponse: `(error: unknown) => import('agent-device/contracts').DaemonResponse`,
6373
};
6474

65-
// Pre-existing gaps this check surfaced that are out of scope for #1463 to fix
66-
// (they predate this PR and touch unrelated public API surface, not the new
67-
// examples): flagged for the maintainer rather than silently patched. Remove
68-
// an entry once the underlying gap is closed.
69-
const KNOWN_DOC_GAPS = [
70-
// "Remote Metro helpers": prepareRemoteMetro/reloadRemoteMetro/stopMetroTunnel
71-
// aren't exported from `agent-device/metro`, and resolveRemoteConfigProfile
72-
// isn't exported from `agent-device/remote-config` — the doc documents a
73-
// public surface neither subpath's src/sdk/*.ts re-exports today.
74-
`Module '"agent-device/metro"' has no exported member 'prepareRemoteMetro'`,
75-
`Module '"agent-device/metro"' has no exported member 'reloadRemoteMetro'`,
76-
`Module '"agent-device/metro"' has no exported member 'stopMetroTunnel'`,
77-
`'"agent-device/remote-config"' has no exported member named 'resolveRemoteConfigProfile'`,
78-
// "Web sessions" / audio probe: NetworkOptions/AudioOptions have no `platform`
79-
// field, even though the CLI's `network`/`audio` commands accept `--platform`
80-
// — the typed client's contracts are narrower than the CLI surface here.
81-
`'platform' does not exist in type 'NetworkOptions'`,
82-
`'platform' does not exist in type 'AudioOptions'`,
83-
];
84-
8575
function writeSnippetProgram(snippets: string[]): string {
8676
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'client-api-doc-snippets-'));
8777
fs.writeFileSync(path.join(tmpDir, 'package.json'), '{"type":"module"}\n');
@@ -112,44 +102,46 @@ function writeSnippetProgram(snippets: string[]): string {
112102
}
113103

114104
function runTsc(tmpDir: string): string {
115-
try {
116-
execFileSync(
117-
TSC_BIN,
118-
['--noEmit', '-p', path.join(tmpDir, 'tsconfig.json'), '--pretty', 'false'],
119-
{
120-
encoding: 'utf8',
121-
cwd: tmpDir,
122-
},
123-
);
124-
return '';
125-
} catch (error) {
126-
const { stdout, stderr } = error as { stdout?: string; stderr?: string };
127-
return `${stdout ?? ''}${stderr ?? ''}`;
128-
}
105+
const result = runCmdSync(
106+
TSC_BIN,
107+
['--noEmit', '-p', path.join(tmpDir, 'tsconfig.json'), '--pretty', 'false'],
108+
{ cwd: tmpDir, allowFailure: true },
109+
);
110+
return `${result.stdout}${result.stderr}`;
129111
}
130112

131-
// One auto-stub pass: for every "Cannot find name 'X'" diagnostic, declare `X`
132-
// (typed precisely if it's a known SDK-derived continuation, `any` otherwise)
133-
// at the top of that snippet file and recompile. A missing/renamed *import*
134-
// binding is a different diagnostic code and is never suppressed this way.
135-
function stubFreeNamesAndRecompile(tmpDir: string, firstPassOutput: string): string {
113+
// Finds every "Cannot find name 'X'" diagnostic where `X` is in
114+
// KNOWN_FREE_NAME_STUB_TYPES, grouped by snippet file. Any other free name —
115+
// anything not in that explicit allowlist — is deliberately left out here, so
116+
// its diagnostic survives untouched into the final output and fails the
117+
// check. A missing/renamed *import* binding is a different diagnostic code
118+
// and is never matched by this regex either way.
119+
function findKnownFreeNamesByFile(tscOutput: string): Map<string, Set<string>> {
136120
const freeNamesByFile = new Map<string, Set<string>>();
137-
for (const line of firstPassOutput.split('\n')) {
121+
for (const line of tscOutput.split('\n')) {
138122
const match = /^(snippet-\d+\.ts)\(\d+,\d+\): error TS2304: Cannot find name '([^']+)'/.exec(
139123
line,
140124
);
141-
if (match?.[1] && match[2]) {
125+
const name = match?.[2];
126+
if (match?.[1] && name && name in KNOWN_FREE_NAME_STUB_TYPES) {
142127
const names = freeNamesByFile.get(match[1]) ?? new Set<string>();
143-
names.add(match[2]);
128+
names.add(name);
144129
freeNamesByFile.set(match[1], names);
145130
}
146131
}
132+
return freeNamesByFile;
133+
}
134+
135+
// One auto-stub pass: declare each known free name at the top of its snippet
136+
// file (typed per KNOWN_FREE_NAME_STUB_TYPES) and recompile.
137+
function stubFreeNamesAndRecompile(tmpDir: string, firstPassOutput: string): string {
138+
const freeNamesByFile = findKnownFreeNamesByFile(firstPassOutput);
147139
if (freeNamesByFile.size === 0) return firstPassOutput;
148140

149141
for (const [fileName, names] of freeNamesByFile) {
150142
const filePath = path.join(tmpDir, fileName);
151143
const stubs = [...names]
152-
.map((name) => `declare const ${name}: ${KNOWN_CONTINUATION_STUB_TYPES[name] ?? 'any'};`)
144+
.map((name) => `declare const ${name}: ${KNOWN_FREE_NAME_STUB_TYPES[name]};`)
153145
.join('\n');
154146
fs.writeFileSync(
155147
filePath,
@@ -163,8 +155,7 @@ function compileDocSnippets(snippets: string[]): string[] {
163155
const tmpDir = writeSnippetProgram(snippets);
164156
try {
165157
const output = stubFreeNamesAndRecompile(tmpDir, runTsc(tmpDir));
166-
const diagnosticLines = output.split('\n').filter((line) => /error TS\d+:/.test(line));
167-
return diagnosticLines.filter((line) => !KNOWN_DOC_GAPS.some((gap) => line.includes(gap)));
158+
return output.split('\n').filter((line) => /error TS\d+:/.test(line));
168159
} finally {
169160
fs.rmSync(tmpDir, { recursive: true, force: true });
170161
}
@@ -183,8 +174,21 @@ test("every fenced ```ts snippet in client-api.md compiles against agent-device'
183174
[],
184175
`A \`\`\`ts snippet in ${CLIENT_API_DOC_PATH} no longer compiles against agent-device's real ` +
185176
`exports:\n${failures.join('\n')}\n` +
186-
'Fix the snippet to match the current API, or update the SDK if the doc was right and the ' +
187-
'API regressed. If this is a new, deliberately out-of-scope gap, add it to KNOWN_DOC_GAPS ' +
188-
'with a comment explaining why.',
177+
'Fix the snippet to match the current API, or fix the actual exported contract if the doc ' +
178+
'was right and the API regressed.',
179+
);
180+
});
181+
182+
test('rejects an unrecognized free identifier instead of silently stubbing it (e.g. a typo of `client`)', () => {
183+
const failures = compileDocSnippets([
184+
"import { createAgentDeviceClient } from 'agent-device';\n\n" +
185+
"const client = createAgentDeviceClient({ session: 'qa-ios' });\n" +
186+
"await cliet.apps.open({ app: 'com.example.app', platform: 'ios' });\n",
187+
]);
188+
assert.ok(
189+
failures.some((line) => line.includes("Cannot find name 'cliet'")),
190+
`Expected a "Cannot find name 'cliet'" failure for an unrecognized free identifier, got: ` +
191+
`${JSON.stringify(failures)}. If this fails, stubFreeNamesAndRecompile is stubbing names ` +
192+
'outside its explicit allowlist again.',
189193
);
190194
});

‎website/docs/docs/client-api.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,12 @@ Public subpath API exposed for Node consumers:
2222
- `buildBundleUrl(baseUrl, platform)`
2323
- `normalizeBaseUrl(baseUrl)`
2424
- `resolveRuntimeTransport(runtime)`
25+
- `prepareMetroRuntime(options?)`, `reloadMetro(options?)`, `stopMetroTunnel(options)`
2526
- types: `MetroBridgeDescriptor`, `MetroTunnelRequestMessage`, `MetroTunnelResponseMessage`
2627
- `agent-device/batch`
2728
- `runBatch(req, sessionName, invoke)`
2829
- `agent-device/remote-config`
30+
- `resolveRemoteConfigProfile(options)`
2931
- types: `RemoteConfigProfile`
3032
- `agent-device/contracts`
3133
- `centerOfRect(rect)`
@@ -342,6 +344,8 @@ async function handleBatch(req: BatchRequest): Promise<DaemonResponse> {
342344
## Android `installFromSource()`
343345

344346
```ts
347+
import { createAgentDeviceClient } from 'agent-device';
348+
345349
const androidClient = createAgentDeviceClient({ session: 'qa-android' });
346350

347351
const installed = await androidClient.apps.installFromSource({
@@ -403,15 +407,15 @@ Direct Android `.apk` and `.aab` URL sources can still resolve package identity
403407
## Remote Metro helpers
404408

405409
```ts
406-
import { prepareRemoteMetro, reloadRemoteMetro, stopMetroTunnel } from 'agent-device/metro';
410+
import { prepareMetroRuntime, reloadMetro, stopMetroTunnel } from 'agent-device/metro';
407411
import { resolveRemoteConfigProfile } from 'agent-device/remote-config';
408412

409413
const remoteConfig = resolveRemoteConfigProfile({
410414
configPath: './agent-device.remote.json',
411415
cwd: process.cwd(),
412416
});
413417

414-
const prepared = await prepareRemoteMetro({
418+
const prepared = await prepareMetroRuntime({
415419
projectRoot: remoteConfig.profile.metroProjectRoot!,
416420
kind: remoteConfig.profile.metroKind ?? 'auto',
417421
proxyBaseUrl: remoteConfig.profile.metroProxyBaseUrl,
@@ -421,12 +425,12 @@ const prepared = await prepareRemoteMetro({
421425
runId: remoteConfig.profile.runId!,
422426
leaseId: remoteConfig.profile.leaseId!,
423427
},
424-
profileKey: remoteConfig.resolvedPath,
428+
companionProfileKey: remoteConfig.resolvedPath,
425429
});
426430

427431
console.log(prepared.iosRuntime, prepared.androidRuntime);
428432

429-
await reloadRemoteMetro({
433+
await reloadMetro({
430434
runtime: prepared.iosRuntime,
431435
});
432436

0 commit comments

Comments
 (0)