Skip to content

Commit 3c09bb2

Browse files
thymikeeclaude
andauthored
fix(daemon): reject unsupported raw --save-script flags at request ingress (#1501)
* fix(daemon): reject unsupported raw --save-script flags at request ingress `flags.saveScript` arms session-script publication the moment a successful handler records the request's action, so any recordable command reaching a handler with the raw flag set could arm publication and write a `.ad` artifact — `record stop` immediately through record-only cleanup, `trace` and the interaction commands later on close. The CLI/Node/MCP surfaces only emit the flag for its released owners, so that arming path was reachable only by a hand-built wire request or a free-form `batch` step. Close it at the daemon request seam (`createRequestHandler`), the single entry point both the socket and HTTP transports funnel through, and ahead of admission, session lookup, device resolution, and handler dispatch. Ownership is declared on the command descriptor (`saveScriptFlagOwner`) rather than re-stated as a string set, so `open`, `close`, and `replay` keep the flag and everything else is rejected with an ADR 0010 shaped `INVALID_ARGS` carrying a hint, diagnosticId, and logPath. With that path pinned closed, a record-only session can no longer carry `recordSession`, so `record stop`'s immediate `writeSessionLog` could only ever be a no-op; delete it. Refs #1478. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 * test(daemon): assert ADR 0010 diagnostics on save-script rejection The HTTP test helper projected only code/message/hint, so `diagnosticId` and `logPath` could regress while the transport tests stayed green. Preserve both in the helper and assert on each transport that `diagnosticId` is present, `logPath` is present and exists, and the log records `save_script_flag_rejected`. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8 --------- Co-authored-by: Claude <[email protected]>
1 parent 0ee2a86 commit 3c09bb2

8 files changed

Lines changed: 466 additions & 6 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## Unreleased
44

5+
- `--save-script` is now accepted only by the commands that declare it — `open`, `close`, and `replay`. A hand-built daemon request (or a `batch` step) that set `saveScript` on any other command, such as `record` or `trace`, used to arm script publication and could write a `.ad` artifact; it is now rejected with `INVALID_ARGS` before the request reaches admission, the device, or any handler. CLI, Node, and MCP usage of `--save-script` on its documented commands is unchanged.
56
- `diff screenshot` no longer runs the retired best-effort OCR and non-text analyzers. Their optional `ocr` and `nonTextDeltas` fields remain in the result type for source compatibility but are no longer emitted; use the baseline/current images and diff artifact with vision for qualitative interpretation.
67
- Breaking: removed the deprecated `--session-locked` and `--session-lock-conflicts` flags. Use `--session-lock reject|strip` instead; passing either old flag now fails with `Unknown flag: ... Use --session-lock reject|strip instead.`
78
- Breaking: removed the `replay export --format` flag. `replay export` always writes Maestro YAML.

‎src/core/command-descriptor/registry.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,7 @@ export const RAW_COMMAND_DESCRIPTORS = [
552552
refFrameEffect: 'delegated',
553553
sessionKind: 'replay',
554554
skipSessionlessProviderDevice: isShardedTestRequest,
555+
saveScriptFlagOwner: true,
555556
},
556557
// Replay durations are script-dependent; --timeout bounds the envelope.
557558
timeoutPolicy: { ...DEFAULT_TIMEOUT_POLICY, budget: { source: 'flag' } },
@@ -703,6 +704,7 @@ export const RAW_COMMAND_DESCRIPTORS = [
703704
route: 'session',
704705
refFrameEffect: 'may-invalidate',
705706
allowSessionlessDefaultDevice: allowAnyDeviceSessionless,
707+
saveScriptFlagOwner: true,
706708
},
707709
dispatch: {},
708710
capability: VEGA_APP_RUNTIME_CAPABILITY,
@@ -742,7 +744,12 @@ export const RAW_COMMAND_DESCRIPTORS = [
742744
catalog: { group: 'public' },
743745
recordsSessionAction: true,
744746
recordingEffect: 'mutates-app',
745-
daemon: { route: 'session', refFrameEffect: 'may-invalidate', allowInvalidRecording: true },
747+
daemon: {
748+
route: 'session',
749+
refFrameEffect: 'may-invalidate',
750+
allowInvalidRecording: true,
751+
saveScriptFlagOwner: true,
752+
},
746753
dispatch: {},
747754
capability: VEGA_APP_RUNTIME_CAPABILITY,
748755
timeoutPolicy: DEFAULT_TIMEOUT_POLICY,
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
/**
2+
* #1478 (P4-pre): `flags.saveScript` is accepted only on the released
3+
* `--save-script` flag owners. Everything else is rejected at the daemon
4+
* request seam before any admission, session, or device work.
5+
*/
6+
import { expect, test } from 'vitest';
7+
import { listSaveScriptFlagOwnerCommands, ownsSaveScriptFlag } from '../daemon-command-registry.ts';
8+
import {
9+
SAVE_SCRIPT_FLAG_OWNER_COMMANDS,
10+
unsupportedSaveScriptFlagResponse,
11+
} from '../request-save-script-policy.ts';
12+
import type { DaemonRequest } from '../types.ts';
13+
14+
function request(command: string, flags: DaemonRequest['flags']): DaemonRequest {
15+
return { token: 'token', session: 'default', command, positionals: [], flags };
16+
}
17+
18+
test('the flag owners are exactly the released open/close/replay surface', () => {
19+
expect(listSaveScriptFlagOwnerCommands()).toEqual(['close', 'open', 'replay']);
20+
expect(SAVE_SCRIPT_FLAG_OWNER_COMMANDS).toEqual(['close', 'open', 'replay']);
21+
for (const command of SAVE_SCRIPT_FLAG_OWNER_COMMANDS) {
22+
expect(ownsSaveScriptFlag(command)).toBe(true);
23+
}
24+
// `test` runs replay scripts but never declares `--save-script`, and the
25+
// internal publication command carries its path/force as positionals+flags of
26+
// its own — neither may arm through the raw flag.
27+
expect(ownsSaveScriptFlag('test')).toBe(false);
28+
expect(ownsSaveScriptFlag('session_save_script')).toBe(false);
29+
});
30+
31+
test('owner commands and flag-free requests pass the seam untouched', () => {
32+
for (const command of SAVE_SCRIPT_FLAG_OWNER_COMMANDS) {
33+
expect(unsupportedSaveScriptFlagResponse(request(command, { saveScript: true }))).toBe(
34+
undefined,
35+
);
36+
expect(unsupportedSaveScriptFlagResponse(request(command, { saveScript: './flow.ad' }))).toBe(
37+
undefined,
38+
);
39+
}
40+
expect(unsupportedSaveScriptFlagResponse(request('record', {}))).toBe(undefined);
41+
expect(unsupportedSaveScriptFlagResponse(request('record', undefined))).toBe(undefined);
42+
});
43+
44+
test.each(['record', 'trace', 'click', 'fill', 'snapshot', 'test', 'session_save_script'])(
45+
'raw saveScript on %s is rejected with an ADR 0010 shaped INVALID_ARGS error',
46+
(command) => {
47+
const response = unsupportedSaveScriptFlagResponse(request(command, { saveScript: true }));
48+
49+
expect(response?.ok).toBe(false);
50+
if (!response || response.ok) return;
51+
expect(response.error.code).toBe('INVALID_ARGS');
52+
expect(response.error.message).toBe('--save-script is supported only by close, open, replay.');
53+
// The per-code default hint ("check command arguments") would misdirect, so
54+
// the rejection names the surfaces that actually publish a script.
55+
expect(response.error.hint).toMatch(/session save-script/);
56+
},
57+
);
58+
59+
test('presence is rejected, not truthiness — a raw false is unsupported too', () => {
60+
const response = unsupportedSaveScriptFlagResponse(request('record', { saveScript: false }));
61+
62+
expect(response?.ok).toBe(false);
63+
if (!response || response.ok) return;
64+
expect(response.error.code).toBe('INVALID_ARGS');
65+
});
Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
/**
2+
* #1478 (P4-pre): raw-wire counterfactuals for the `flags.saveScript` seam.
3+
*
4+
* The CLI/Node/MCP surfaces only ever emit `--save-script` for its released
5+
* owners, so the arming path is reachable only by a hand-built request. Both
6+
* daemon transports funnel through the SAME request handler
7+
* (`createRequestHandler`, wired into `createSocketServer` and
8+
* `createDaemonHttpServer` by `daemon-runtime.ts`), so the rejection is pinned
9+
* here on both wires: an unsupported command carrying the flag must not reach
10+
* admission or device work, must not arm publication, and must not leave a
11+
* `.ad` artifact behind.
12+
*/
13+
import fs from 'node:fs';
14+
import net from 'node:net';
15+
import os from 'node:os';
16+
import path from 'node:path';
17+
import { afterEach, expect, test } from 'vitest';
18+
import { LeaseRegistry } from '../lease-registry.ts';
19+
import { createRequestHandler } from '../request-router.ts';
20+
import { SessionStore } from '../session-store.ts';
21+
import { createDaemonHttpServer } from '../server/http-server.ts';
22+
import { createSocketServer, listenNetServer } from '../server/transport.ts';
23+
import type { DaemonInvokeFn, DaemonResponse, SessionState } from '../types.ts';
24+
import { makeIosSession } from '../../__tests__/test-utils/index.ts';
25+
import {
26+
closeLoopbackServer,
27+
listenOnLoopback,
28+
skipWhenLoopbackUnavailable,
29+
} from '../../__tests__/test-utils/loopback.ts';
30+
31+
const TOKEN = 'save-script-transport-token';
32+
const SESSION = 'save-script-transport';
33+
const UNSUPPORTED_MESSAGE = '--save-script is supported only by close, open, replay.';
34+
35+
type WireRequest = {
36+
token?: string;
37+
session?: string;
38+
command: string;
39+
positionals?: string[];
40+
flags?: Record<string, unknown>;
41+
};
42+
43+
type Harness = {
44+
root: string;
45+
sessionStore: SessionStore;
46+
session: SessionState;
47+
handleRequest: DaemonInvokeFn;
48+
};
49+
50+
const roots: string[] = [];
51+
52+
afterEach(() => {
53+
for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
54+
});
55+
56+
function setup(): Harness {
57+
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-save-script-transport-'));
58+
roots.push(root);
59+
const sessionStore = new SessionStore(path.join(root, 'sessions'));
60+
const session = makeIosSession(SESSION);
61+
sessionStore.set(SESSION, session);
62+
const handleRequest = createRequestHandler({
63+
logPath: path.join(root, 'daemon.log'),
64+
token: TOKEN,
65+
sessionStore,
66+
leaseRegistry: new LeaseRegistry(),
67+
trackDownloadableArtifact: () => 'artifact-id',
68+
});
69+
return { root, sessionStore, session, handleRequest };
70+
}
71+
72+
async function sendOverSocket(
73+
handleRequest: DaemonInvokeFn,
74+
request: WireRequest,
75+
): Promise<DaemonResponse> {
76+
const server = createSocketServer(handleRequest);
77+
try {
78+
const port = await listenNetServer(server);
79+
const socket = net.createConnection({ host: '127.0.0.1', port });
80+
try {
81+
await new Promise<void>((resolve, reject) => {
82+
socket.once('connect', () => resolve());
83+
socket.once('error', reject);
84+
});
85+
return await new Promise<DaemonResponse>((resolve, reject) => {
86+
let buffer = '';
87+
socket.setEncoding('utf8');
88+
socket.on('data', (chunk: string) => {
89+
buffer += chunk;
90+
const newline = buffer.indexOf('\n');
91+
if (newline === -1) return;
92+
resolve(JSON.parse(buffer.slice(0, newline)) as DaemonResponse);
93+
});
94+
socket.once('error', reject);
95+
socket.write(`${JSON.stringify({ token: TOKEN, session: SESSION, ...request })}\n`);
96+
});
97+
} finally {
98+
socket.destroy();
99+
}
100+
} finally {
101+
await closeLoopbackServer(server);
102+
}
103+
}
104+
105+
async function sendOverHttp(
106+
handleRequest: DaemonInvokeFn,
107+
request: WireRequest,
108+
): Promise<DaemonResponse> {
109+
const server = await createDaemonHttpServer({ handleRequest, token: TOKEN });
110+
try {
111+
const port = await listenOnLoopback(server);
112+
const response = await fetch(`http://127.0.0.1:${port}/rpc`, {
113+
method: 'POST',
114+
headers: { 'content-type': 'application/json' },
115+
body: JSON.stringify({
116+
jsonrpc: '2.0',
117+
id: 'req-save-script',
118+
method: 'agent_device.command',
119+
params: { token: TOKEN, session: SESSION, ...request },
120+
}),
121+
});
122+
const body = (await response.json()) as {
123+
result?: { ok: boolean; data?: Record<string, unknown> };
124+
// Project the ADR 0010 diagnostics fields too: dropping them here would
125+
// let `diagnosticId`/`logPath` regress while these tests stayed green.
126+
error?: {
127+
data?: {
128+
code?: string;
129+
message?: string;
130+
hint?: string;
131+
diagnosticId?: string;
132+
logPath?: string;
133+
};
134+
};
135+
};
136+
if (body.error?.data) {
137+
const { code, message, hint, diagnosticId, logPath } = body.error.data;
138+
return {
139+
ok: false,
140+
error: { code: code ?? 'UNKNOWN', message: message ?? '', hint, diagnosticId, logPath },
141+
};
142+
}
143+
return (body.result ?? {
144+
ok: false,
145+
error: { code: 'UNKNOWN', message: 'no result' },
146+
}) as DaemonResponse;
147+
} finally {
148+
await closeLoopbackServer(server);
149+
}
150+
}
151+
152+
const TRANSPORTS = [
153+
['socket', sendOverSocket],
154+
['http', sendOverHttp],
155+
] as const satisfies readonly (readonly [
156+
string,
157+
(handleRequest: DaemonInvokeFn, request: WireRequest) => Promise<DaemonResponse>,
158+
])[];
159+
160+
function listAdArtifacts(root: string): string[] {
161+
return fs
162+
.readdirSync(root, { recursive: true, encoding: 'utf8' })
163+
.filter((entry) => entry.endsWith('.ad'));
164+
}
165+
166+
for (const [transport, send] of TRANSPORTS) {
167+
test(`${transport}: raw saveScript on a recordable command never arms or writes a script`, async (t) => {
168+
if (await skipWhenLoopbackUnavailable(t)) return;
169+
const { root, sessionStore, session, handleRequest } = setup();
170+
171+
const rejected = await send(handleRequest, {
172+
command: 'trace',
173+
positionals: ['start'],
174+
flags: { saveScript: `${root}/forged.ad` },
175+
});
176+
177+
expect(rejected.ok).toBe(false);
178+
if (rejected.ok) return;
179+
expect(rejected.error.code).toBe('INVALID_ARGS');
180+
expect(rejected.error.message).toBe(UNSUPPORTED_MESSAGE);
181+
expect(rejected.error.hint).toMatch(/session save-script/);
182+
// ADR 0010 decision 6: a failed request stays traceable. Asserted on both
183+
// transports so the HTTP projection cannot silently drop these again.
184+
expect(rejected.error.diagnosticId).toBeTruthy();
185+
expect(rejected.error.logPath).toBeTruthy();
186+
const rejectionLogPath = rejected.error.logPath as string;
187+
expect(fs.existsSync(rejectionLogPath)).toBe(true);
188+
expect(fs.readFileSync(rejectionLogPath, 'utf8')).toMatch(/save_script_flag_rejected/);
189+
190+
// No handler work: the trace never started and no action was recorded.
191+
expect(session.trace).toBe(undefined);
192+
expect(session.actions).toEqual([]);
193+
// No arming: neither the recording marker nor the publication target moved.
194+
expect(session.recordSession).toBe(undefined);
195+
expect(session.saveScriptPath).toBe(undefined);
196+
// No artifact: the write a later close/teardown would attempt publishes nothing.
197+
expect(sessionStore.writeSessionLog(session)).toEqual({ written: false });
198+
expect(listAdArtifacts(root)).toEqual([]);
199+
expect(fs.existsSync(path.join(root, 'forged.ad'))).toBe(false);
200+
201+
// Counterfactual: the very same request without the flag does reach the
202+
// handler and does record its action, so the rejection above is the flag.
203+
const accepted = await send(handleRequest, { command: 'trace', positionals: ['start'] });
204+
expect(accepted.ok).toBe(true);
205+
expect(session.trace?.outPath).toMatch(/\.trace\.log$/);
206+
expect(session.actions.map((action) => action.command)).toEqual(['trace']);
207+
expect(session.recordSession).toBe(undefined);
208+
expect(listAdArtifacts(root)).toEqual([]);
209+
});
210+
211+
test(`${transport}: the rejection lands before admission`, async (t) => {
212+
if (await skipWhenLoopbackUnavailable(t)) return;
213+
const { handleRequest } = setup();
214+
215+
// `sessionIsolation: 'tenant'` without a tenant is rejected by
216+
// `scopeRequestSession`, the first step of request admission. Getting the
217+
// save-script message instead proves the flag seam runs ahead of it.
218+
const response = await send(handleRequest, {
219+
command: 'record',
220+
positionals: ['stop'],
221+
flags: { saveScript: true, sessionIsolation: 'tenant' },
222+
});
223+
224+
expect(response.ok).toBe(false);
225+
if (response.ok) return;
226+
expect(response.error.message).toBe(UNSUPPORTED_MESSAGE);
227+
expect(response.error.message).not.toMatch(/tenant/);
228+
});
229+
230+
test(`${transport}: the released owners still carry the flag to their handlers`, async (t) => {
231+
if (await skipWhenLoopbackUnavailable(t)) return;
232+
const { handleRequest } = setup();
233+
234+
// `replay` is a flag owner, so the seam lets it through and the request
235+
// fails only on its own missing-path validation, downstream of admission.
236+
const response = await send(handleRequest, {
237+
command: 'replay',
238+
positionals: [],
239+
flags: { saveScript: true },
240+
});
241+
242+
expect(response.ok).toBe(false);
243+
if (response.ok) return;
244+
expect(response.error.message).toBe('replay requires a path');
245+
});
246+
}
247+
248+
test('a batch step cannot smuggle the flag onto a non-owner command', async (t) => {
249+
if (await skipWhenLoopbackUnavailable(t)) return;
250+
const { root, session, handleRequest } = setup();
251+
252+
// Batch step flags are free-form passthrough into the same request entry
253+
// point, so they are the third face of the same raw arming path — the step's
254+
// nested request meets the seam exactly like a top-level one.
255+
const response = await sendOverSocket(handleRequest, {
256+
command: 'batch',
257+
positionals: [],
258+
flags: {
259+
batchSteps: [
260+
{ command: 'trace', positionals: ['start'], flags: { saveScript: `${root}/forged.ad` } },
261+
],
262+
},
263+
});
264+
265+
expect(response.ok).toBe(false);
266+
if (response.ok) return;
267+
expect(response.error.message).toMatch(UNSUPPORTED_MESSAGE);
268+
expect(session.trace).toBe(undefined);
269+
expect(session.recordSession).toBe(undefined);
270+
expect(listAdArtifacts(root)).toEqual([]);
271+
});
272+
273+
test('an owner-armed session still records its target and publishes its script', () => {
274+
const { root, sessionStore, session } = setup();
275+
const target = path.join(root, 'published.ad');
276+
277+
// What `open`/`close --save-script` do once past the seam: arm the session,
278+
// then publish at teardown. Unchanged by the ingress rejection.
279+
sessionStore.recordAction(session, {
280+
command: 'open',
281+
positionals: ['Example'],
282+
flags: { saveScript: target },
283+
result: { session: SESSION },
284+
});
285+
expect(session.recordSession).toBe(true);
286+
expect(session.saveScriptPath).toBe(target);
287+
288+
const result = sessionStore.writeSessionLog(session);
289+
expect(result).toEqual({ written: true, path: target, actionCount: 1 });
290+
expect(fs.readFileSync(target, 'utf8')).toMatch(/^open /m);
291+
});

0 commit comments

Comments
 (0)