Skip to content

Commit 7f75a37

Browse files
MatAtBreadclaude
andcommitted
feat(function-tools): bound synchronous work in model-authored code
Session 18da1564 froze the daemon for 260s: a tool_function lambda's `while` never advanced, and blocked the one event loop every session, frontend and schedule shares. It ended only because the array it grew hit the length limit. - plugin-api: optional `FunctionRunner` service — compiles stripped code into an async function whose synchronous execution is bounded. Absent ⇒ bodies run directly, as before (the browser, or an embedder without one). - cli: registers a node:vm runner in the host's own context. A body doing more than `function_timeout_ms` (matbot.yaml; default 10s) of work before its first await is stopped with an error naming the likely runaway loop. `0` registers no runner, for testing, and warns at boot. Nested runs each take their own call; dropped promises are marked handled so a stopped run cannot crash the process. - function-tools: lambdas, defined functions and packages compile through the runner; a package export is called inside the timed run. `runFunction` stops waiting when its call is aborted. - plugin-api: `invokeTool` refuses to start a tool on an aborted signal, so a body looping over tool calls stops once its turn is cancelled. Not covered: a loop after an await. The design that covers it (a context per run in afterEvaluate mode) aborts the process when async hooks are enabled (nodejs/node#38503, closed unfixed). Web bundle rebuilt (it bakes core and function-tools). Co-Authored-By: Claude Opus 5 <[email protected]>
1 parent 376cdd3 commit 7f75a37

15 files changed

Lines changed: 344 additions & 36 deletions

File tree

‎.changeset/function-runner.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
"@matatbread/matbot-plugin-api": patch
3+
"@matatbread/matbot-core": patch
4+
"@matatbread/matbot-function-tools": patch
5+
"@matatbread/matbot-cli": patch
6+
---
7+
8+
Bound model-authored code: an optional `FunctionRunner` service (registered by the CLI over `node:vm`) stops a `tool_function` that does more than 10s of synchronous work without awaiting, instead of freezing the daemon. The limit is `function_timeout_ms` in `matbot.yaml`; `0` registers no runner. `invokeTool` refuses to start a tool on an aborted signal, and `runFunction` stops waiting when its call is aborted.

‎CHANGELOG.md‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,31 @@ filled**, and **Bug fixes** cover `core` (the contract consumers depend on);
99
**Optional** covers new or updated plugins, frontends, and apps — more likely to
1010
churn and less likely to affect a consumer who doesn't use them.
1111

12-
## Unreleased
12+
## 0.4.15
13+
14+
### API gaps filled
15+
16+
- **`FunctionRunner`** — an optional `MatbotServices` member: a host-supplied compiler for model-authored
17+
code whose *synchronous* execution is bounded, since a loop that never awaits otherwise freezes every
18+
session sharing the event loop. Only stretches between awaits are bounded, so slow tool calls are
19+
unaffected. Absent ⇒ unbounded, as before (and as the browser must be).
20+
21+
### Bug fixes
22+
23+
- **`invokeTool`** refuses to start a tool once `opts.signal` has aborted, so a composed body looping over
24+
`await tool.x()` stops calling tools when its turn is cancelled rather than leaving each callee to decide.
1325

1426
### Optional
1527

28+
- **`function-tools`** — lambdas, defined functions and packages compile through `services.FunctionRunner`
29+
when present; a package export is now called inside that bounded run. `runFunction` stops waiting when
30+
its call is aborted and reports it cancelled.
31+
- **`cli`** — registers a `node:vm` `FunctionRunner`: a `tool_function` doing more than 10s of synchronous
32+
work before its first await is stopped with an error naming the likely runaway loop, instead of freezing
33+
the daemon. The limit is `function_timeout_ms` in `matbot.yaml`; `0` registers no runner at all (bodies
34+
run unbounded, as before), for testing, and warns at boot. A loop after an await is not covered: the
35+
design that covers it aborts the process when async hooks are enabled (nodejs/node#38503).
36+
1637
- **`function-tools`** — `tool_function { action: 'package', name, definition }` defines a **package**: one
1738
TypeScript module whose `export`ed functions become tools named `<package>__<function>`, and whose other
1839
declarations (helpers, types, constants) are private and never registered — so an internal helper no

‎apps/cli/src/function-runner.ts‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
import vm from 'node:vm';
2+
import type { FunctionRunner } from '@matatbread/matbot-core';
3+
4+
// The default, when `function_timeout_ms` is absent from matbot.yaml. Far past any legitimate stretch of
5+
// computation between two awaits, and short enough that the daemon is back before anyone concludes it died.
6+
export const FUNCTION_SYNC_LIMIT_MS = 10_000;
7+
8+
const PENDING = 'matbot.functionRunner.pending';
9+
10+
// A vm timeout bounds only code started by a vm script, so each call runs through this one: it pops the
11+
// thunk pushed just before it runs. LIFO is what makes a nested call safe — one body's tool call can start
12+
// another function before its first await, inside the outer script — each run taking its own thunk.
13+
const invoke = new vm.Script(`globalThis[Symbol.for(${JSON.stringify(PENDING)})].pop()()`, { filename: 'tool_function' });
14+
15+
/**
16+
* A {@link FunctionRunner} over `node:vm`: runs in THIS context, so a body keeps the globals it has today
17+
* (`fetch`, timers, `console`), and bounds each call's synchronous run from its start to its first await.
18+
*
19+
* Code after an await is NOT bounded, because a vm timeout covers only the evaluation it is given and an
20+
* async body's evaluation ends at that await. Covering it needs a context per run in `afterEvaluate` mode,
21+
* with continuations drained inside timed evaluations — which works, but a timeout that lands during that
22+
* drain aborts the whole process whenever async hooks are enabled (nodejs/node#38503, closed unfixed).
23+
* matbot enables none, but the test runner does and any instrumentation might, and a guard that can kill
24+
* the daemon is worse than the freeze it prevents. An aborted call at least stops such a body's tool calls.
25+
*/
26+
export function createVmFunctionRunner(limitMs = FUNCTION_SYNC_LIMIT_MS): FunctionRunner {
27+
const slot = globalThis as unknown as Record<symbol, Array<() => Promise<unknown>> | undefined>;
28+
const stack = (slot[Symbol.for(PENDING)] ??= []);
29+
return {
30+
compile(params, body) {
31+
const fn = vm.runInThisContext(`(async function (${params.join(', ')}) {\n${body}\n})`, { filename: 'tool_function' }) as
32+
(...args: unknown[]) => Promise<unknown>;
33+
return (...args) => {
34+
const depth = stack.length;
35+
let started: Promise<unknown> | undefined;
36+
stack.push(() => (started = fn(...args)));
37+
let pending: Promise<unknown>;
38+
try {
39+
pending = invoke.runInThisContext({ timeout: limitMs }) as Promise<unknown>;
40+
} catch (e) {
41+
if ((e as { code?: unknown }).code !== 'ERR_SCRIPT_EXECUTION_TIMEOUT') return Promise.reject(e);
42+
pending = Promise.reject(new Error(`Stopped after ${limitMs / 1000}s of synchronous work without an await — most likely a loop that never ends. A function may compute between awaits, but not for this long.`));
43+
} finally {
44+
// Stopped code skips its own finally blocks, so a thunk pushed inside it may never have been
45+
// popped; trimming to this call's depth keeps the next call from running a stale one.
46+
stack.length = depth;
47+
}
48+
// When a run is stopped, promises it had already created are dropped with it — the body's own
49+
// (which may have adopted a nested run's rejection) and the timeout's. Nobody awaits them, and an
50+
// unhandled rejection would take the whole process down. Marked here, outside the stopped script,
51+
// so this always runs; whoever does await `pending` still sees the rejection.
52+
const noop = (): void => { /* reported through `pending`, or by the enclosing run */ };
53+
started?.catch(noop);
54+
pending.catch(noop);
55+
return pending;
56+
};
57+
},
58+
};
59+
}

‎apps/cli/src/index.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ import { systemPrincipal } from '@matatbread/matbot-core';
3232
import { createAlsPrincipalCarrier } from './principal-als.js';
3333
import { createAlsUsageCarrier } from './usage-als.js';
3434
import { EnvFileVault } from './env-vault.js';
35+
import { createVmFunctionRunner, FUNCTION_SYNC_LIMIT_MS } from './function-runner.js';
3536
import { FilesystemStore } from '@matatbread/matbot-storage-filesystem';
3637
import { FilesystemFileStore } from '@matatbread/matbot-files-node';
3738
import { createBuiltinTools, createProviderTool, classifySpecifier, materializeRemote, remoteDependencyNotes,
@@ -1133,6 +1134,15 @@ async function main(): Promise<void> {
11331134
// proxy, so media follows a StorageBackend swap exactly as every other file does.
11341135
serviceRegistry.set('MediaStore', fileStore);
11351136

1137+
// Model-authored code shares the one event loop with every session and frontend, so a loop in a
1138+
// tool_function that never awaits would freeze the daemon. Seeded, like MediaStore, so a plugin may
1139+
// replace it and unregistering reverts here. `function_timeout_ms: 0` seeds none: bodies then run directly
1140+
// and unbounded — what a host with no runner does — which is kept reachable for testing.
1141+
const functionTimeoutMs = matbotConfig.functionTimeoutMs ?? FUNCTION_SYNC_LIMIT_MS;
1142+
const functionRunner = functionTimeoutMs > 0 ? createVmFunctionRunner(functionTimeoutMs) : undefined;
1143+
if (functionRunner !== undefined) serviceRegistry.set('FunctionRunner', functionRunner);
1144+
else console.warn('[matbot] function_timeout_ms is 0: tool_function bodies run unbounded, and one that loops without awaiting will freeze this process.');
1145+
11361146
// Constructed just after the services object (it closes over services.loadPlugin); exposed via
11371147
// the `run` getter below so frontends submit/observe through one serialiser instead of each
11381148
// calling runSession directly.
@@ -1172,6 +1182,7 @@ async function main(): Promise<void> {
11721182
// Reverts to the host file area rather than vanishing: unloading a plugin that put media on S3
11731183
// should leave attachments working on disk, not silently turn them off until a restart.
11741184
else if (key === 'MediaStore') serviceRegistry.set('MediaStore', fileStore);
1185+
else if (key === 'FunctionRunner' && functionRunner !== undefined) serviceRegistry.set('FunctionRunner', functionRunner);
11751186
else serviceRegistry.delete(key);
11761187
if (key !== 'StorageBackend') { mountTable.markDirty(key as keyof MatbotServices); scheduleEdge(); }
11771188
},
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
import { test } from 'node:test';
2+
import assert from 'node:assert/strict';
3+
import { stripTypeScriptTypes } from 'node:module';
4+
import { buildAsyncFn, runFunction, parsePackage, buildPackageFn, exportFn } from '@matatbread/matbot-function-tools';
5+
import { invokeTool, parseConfig } from '@matatbread/matbot-core';
6+
import type { MatbotMachine, Tool, ToolContext, ToolEvent } from '@matatbread/matbot-plugin-api';
7+
import { createVmFunctionRunner, FUNCTION_SYNC_LIMIT_MS } from '../src/function-runner.ts';
8+
9+
// A tool_function body runs on the daemon's one event loop. 18da1564 (2026-09-15) froze every session for
10+
// 260s on a model-authored `while` whose step never advanced; it ended only because the array it grew hit
11+
// the length limit. These pin the runner that bounds such work, and the abort wiring for loops that await.
12+
13+
const stripper = { strip: (s: string) => stripTypeScriptTypes(s) };
14+
const runner = createVmFunctionRunner(200);
15+
const LIMIT = /synchronous work without an await/;
16+
17+
function machineWith(tools: Tool[] = []): MatbotMachine {
18+
const reg = new Map(tools.map(t => [t.name, t]));
19+
return {
20+
tools: {
21+
list: () => [...reg.values()],
22+
resolve: (n: string) => reg.get(n) ?? null,
23+
register: (t: Tool) => { reg.set(t.name, t); },
24+
remove: (n: string) => reg.delete(n),
25+
},
26+
} as unknown as MatbotMachine;
27+
}
28+
29+
const ctxFor = (signal: AbortSignal = new AbortController().signal): ToolContext => ({
30+
callId: 'c1', session: { id: 's1' }, signal,
31+
prompt: () => Promise.reject(new Error('non-interactive')),
32+
}) as unknown as ToolContext;
33+
34+
async function last(events: AsyncIterable<ToolEvent>): Promise<ToolEvent | undefined> {
35+
let final: ToolEvent | undefined;
36+
for await (const ev of events) final = ev;
37+
return final;
38+
}
39+
40+
const lambda = (src: string) => buildAsyncFn(stripper, src, ['args'], runner);
41+
42+
function pingTool(): Tool & { calls: () => number } {
43+
let calls = 0;
44+
return {
45+
name: 'ping', description: '', inputSchema: { type: 'object' },
46+
executor: { async *execute() { calls++; yield { type: 'result', value: calls }; } },
47+
calls: () => calls,
48+
};
49+
}
50+
const message = (ev: ToolEvent | undefined): string => (ev?.type === 'error' ? ev.message : `not an error: ${JSON.stringify(ev)}`);
51+
52+
test('a loop that never awaits is stopped at the limit, and the daemon carries on', async () => {
53+
const started = Date.now();
54+
const ev = await last(runFunction(machineWith(), ctxFor(), await lambda('(args: {}): Promise<number> { let d = 1; while (d > 0) { d = d; } return d; }'), [{}]));
55+
assert.match(message(ev), LIMIT);
56+
assert.ok(Date.now() - started < 2000, `stopped near the limit, not left to run (${Date.now() - started}ms)`);
57+
assert.equal(await new Promise(r => setTimeout(() => r('alive'), 0)), 'alive', 'the event loop is free again');
58+
});
59+
60+
// Not tested, because not covered: a loop AFTER an await. See createVmFunctionRunner for why the design
61+
// that covers it (a context per run, drained inside timed evaluations) was withdrawn — its timeout aborts
62+
// the process when async hooks are enabled, as they are under this very test runner.
63+
64+
test('a body keeps the globals it had, and its input is its own data', async () => {
65+
const fn = await lambda("(args: { xs: number[] }): Promise<string> { const u = new URL('https://x.test/a?b=1'); await new Promise<void>(r => queueMicrotask(r)); return [typeof fetch, u.searchParams.get('b'), new TextEncoder().encode('hi').length, args.xs instanceof Array, await tool.ping({})].join(','); }");
66+
assert.deepEqual(await last(runFunction(machineWith([pingTool()]), ctxFor(), fn, [{ xs: [1] }])), { type: 'result', value: 'function,1,2,true,1' });
67+
});
68+
69+
test('awaiting is not work: a call slower than the limit completes', async () => {
70+
const ev = await last(runFunction(machineWith(), ctxFor(), await lambda("(args: {}): Promise<string> { await new Promise(r => setTimeout(r, 600)); return 'slow but fine'; }"), [{}]));
71+
assert.deepEqual(ev, { type: 'result', value: 'slow but fine' });
72+
});
73+
74+
test('a package export is bounded too, being called inside the timed run', async () => {
75+
const spin = 'export function spin(args: {}): number { for (;;) {} }';
76+
const spinning = await buildPackageFn(stripper, spin, parsePackage('P', spin), runner);
77+
assert.match(message(await last(runFunction(machineWith(), ctxFor(), exportFn(spinning, 'spin'), [{}]))), LIMIT);
78+
79+
const ok = 'function double(n: number): number { return n * 2; }\nexport function twice(args: { n: number }): number { return double(args.n); }';
80+
const working = await buildPackageFn(stripper, ok, parsePackage('P', ok), runner);
81+
assert.deepEqual(await last(runFunction(machineWith(), ctxFor(), exportFn(working, 'twice'), [{ n: 21 }])), { type: 'result', value: 42 });
82+
});
83+
84+
test('a run started inside another takes its own call, and a loop there is still stopped', async () => {
85+
const inner = runner.compile([], 'return 7;');
86+
assert.equal(await runner.compile(['inner'], 'return inner();')(inner), 7);
87+
const spinning = runner.compile([], 'for (;;) {}');
88+
await assert.rejects(runner.compile(['inner'], 'return inner();')(spinning), LIMIT);
89+
assert.equal(await runner.compile(['a', 'b'], 'return a + b;')(2, 3), 5, 'the runner is usable after a stopped run');
90+
});
91+
92+
test('with no runner the body runs directly, as it did before', async () => {
93+
const fn = await buildAsyncFn(stripper, "(args: {}): Promise<string> { return 'direct'; }", ['args']);
94+
assert.deepEqual(await last(runFunction(machineWith(), ctxFor(), fn, [{}])), { type: 'result', value: 'direct' });
95+
});
96+
97+
test('aborting ends the call, and a body looping over tool calls then stops itself', async () => {
98+
// Awaiting a tool each iteration is what lets it die: after the abort that call is refused and throws.
99+
// A body awaiting only timers would keep running unseen — nothing in-process can stop it — though the
100+
// call still ends on the abort.
101+
const ping = pingTool();
102+
const ac = new AbortController();
103+
const fn = await lambda('(args: {}): Promise<void> { for (;;) { await tool.ping({}); await new Promise(r => setTimeout(r, 5)); } }');
104+
setTimeout(() => ac.abort(), 50);
105+
const started = Date.now();
106+
assert.match(message(await last(runFunction(machineWith([ping]), ctxFor(ac.signal), fn, [{}]))), /Cancelled/);
107+
assert.ok(Date.now() - started < 1000, 'returned promptly after the abort');
108+
const atAbort = ping.calls();
109+
await new Promise(r => setTimeout(r, 60));
110+
assert.equal(ping.calls(), atAbort, 'no tool was started after the abort');
111+
});
112+
113+
test('a tool call on a cancelled call is refused before the tool starts', async () => {
114+
let started = false;
115+
const probe: Tool = {
116+
name: 'probe', description: '', inputSchema: { type: 'object' },
117+
executor: { async *execute() { started = true; yield { type: 'result', value: 1 }; } },
118+
};
119+
const ac = new AbortController();
120+
ac.abort();
121+
assert.throws(() => invokeTool(machineWith([probe]), 'probe', {}, { session: { id: 's1' } as never, signal: ac.signal }), /was not started: the call was cancelled/);
122+
assert.equal(started, false);
123+
});
124+
125+
test('function_timeout_ms comes from matbot.yaml: absent is the default, 0 is a value, a typo is refused', () => {
126+
assert.equal(FUNCTION_SYNC_LIMIT_MS, 10_000);
127+
assert.equal(parseConfig('ephemeral: true\n').functionTimeoutMs, undefined, 'absent ⇒ the host default applies');
128+
assert.equal(parseConfig('function_timeout_ms: 2500\n').functionTimeoutMs, 2500);
129+
// 0 must survive parsing as a value: it is what turns the runner off, not what leaves it at the default.
130+
assert.equal(parseConfig('function_timeout_ms: 0\n').functionTimeoutMs, 0);
131+
for (const bad of ['-1', '1.5', 'soon']) {
132+
assert.throws(() => parseConfig(`function_timeout_ms: ${bad}\n`), /function_timeout_ms/, bad);
133+
}
134+
});

0 commit comments

Comments
 (0)