Repository navigation
[Bug]: Provider handoffs serialize command output that cannot fit the history budget #23820
Description
Area
apps/server / packages/provider-core
Steps to reproduce
- Use main at
22840ab943b8418744b48223411e46ba4fdcef1aon Linux x64, Node 26.12.1. - Build handoff history containing 100 command outputs of 32 KiB each, plus an original brief and recent messages. This represents a long tool-heavy thread.
- Select history with the default 16,000-byte cap using
selectHistoryinpackages/provider-core/src/server/handoffBudget.ts. The reproduction below uses the exact production function.
Expected behavior
Avoid encoding a candidate whose text alone already exceeds the remaining budget.
Actual behavior
Each oversized candidate is rendered and JSON-serialized twice before it is rejected. historicalMessage includes full command output, so provider switches and other context handoffs pay for output that is omitted anyway.
| Same 100 command outputs, 32 KiB each, 16,000-byte budget | Baseline | Patch |
|---|---|---|
| Median (min–max), 21 trials | 5.292 ms (5.096–5.661) | 0.012 ms (0.009–0.018) |
An early UTF-16-length lower-bound check selects the exact same three messages and omits the same 100 command items. The comparison includes the same wrapper, metadata and budget in both versions.
Impact and environment
Extra selector CPU and temporary serialization allocations during handoffs of long tool-heavy threads. Linux x64, Intel i5-14400F, Node 26.12.1. These are warmed synthetic selector measurements, not overall provider-switch latency. Small histories are inexpensive.
Self-contained before/after reproduction
Run from the PR checkout with Node 26.12.1. This loads the exact baseline and patched production functions and checks output equality. No third-party benchmark dependency is needed.
node --input-type=module <<'JS'
import * as assert from 'node:assert/strict';
import * as fs from 'node:fs';
import { execFileSync } from 'node:child_process';
import { stripTypeScriptTypes } from 'node:module';
import { performance } from 'node:perf_hooks';
const base = '22840ab943b8418744b48223411e46ba4fdcef1a';
const file = 'packages/provider-core/src/server/handoffBudget.ts';
async function load(source, name) {
const start = source.indexOf('export function historicalMessage(');
const end = source.indexOf('export function handoffCoverage(', start);
assert.ok(start >= 0 && end > start);
const js = stripTypeScriptTypes(source.slice(start, end));
return import(`data:text/javascript;base64,${Buffer.from(js + `\n//# sourceURL=${name}`).toString('base64')}`);
}
const baseline = await load(execFileSync('git', ['show', `${base}:${file}`], { encoding: 'utf8' }), 'before');
const patched = await load(fs.readFileSync(file, 'utf8'), 'after');
const message = (id, role, text) => ({ itemId: `item:${id}`, role, text, kind: `${role}_message`, threadId: 'thread:fixture', runId: `run:${id}`, providerThreadId: 'provider:fixture', status: 'completed' });
const logs = Array.from({ length: 100 }, (_, i) => {
const line = `test src/component-${i}.test.ts: expected output differs at line ${i + 1}\n`;
return baseline.historicalMessage({ id: `item:command:${i}`, threadId: 'thread:fixture', runId: `run:${Math.floor(i / 5)}`, providerThreadId: 'provider:fixture', status: 'completed', type: 'command_execution', input: `vp test run src/component-${i}.test.ts`, output: line.repeat(Math.ceil(32768 / line.length)).slice(0, 32768), exitCode: 1 });
});
const input = {
messages: [message(0, 'user', 'Keep the API compatible and run the focused tests.'), ...logs, message(10000, 'user', 'Continue investigating the test failures.'), message(10001, 'assistant', 'The failing test is reproduced. Check the captured command output.')],
coverage: 'Provider handoff; omitted history remains retrievable through t3_thread_read.', budget: 16000,
};
const expected = baseline.selectHistory(input);
assert.deepEqual(patched.selectHistory(input), expected);
for (let i = 0; i < 20; i++) { baseline.selectHistory(input); patched.selectHistory(input); }
const samples = { baseline: [], patched: [] };
let sink = 0;
for (let i = 0; i < 21; i++) {
const order = i % 2 ? [['patched', patched], ['baseline', baseline]] : [['baseline', baseline], ['patched', patched]];
for (const [label, parser] of order) {
const start = performance.now();
for (let j = 0; j < 10; j++) sink += parser.selectHistory(input).omittedItems;
samples[label].push((performance.now() - start) / 10);
}
}
const stats = values => {
const sorted = values.toSorted((a, b) => a - b);
return { minMs: sorted[0], medianMs: sorted[10], maxMs: sorted.at(-1), samples: values };
};
console.log(JSON.stringify({ node: process.version, selectedItems: expected.messages.length, omittedItems: expected.omittedItems, baseline: stats(samples.baseline), patched: stats(samples.patched), sink }, null, 2));
JS
Activity
hieu-lee commentedon Oct 30, 2026
Related: #27670 also fixes this.
selectHistorynow skips any candidate whose UTF-16 length alone exceeds the remaining budget, and it sizes the rest arithmetically instead of serializing them. Selection output is identical to before (randomized old-vs-new comparison). It also stops once the budget is exhausted and makes the provider-switch paths read only the historical item types.It overlaps with #23821, which covers the early length check. #27670 includes that check, so whichever lands first, the other will need a small rebase or can be closed.