Skip to content

Commit a1e5192

Browse files
authored
Merge pull request #1637 from RobinBially/hardening/json-rewrite-depth
perf(integrations): depth-cap json configs and harden the serializer walk
2 parents ff674b8 + 23f4349 commit a1e5192

9 files changed

Lines changed: 192 additions & 41 deletions

File tree

‎docs-site/src/content/docs/guides/integrations.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,8 @@ our block — adding an MCP server, a provider of your own — shows as **Update
7171
refreshing merges around your entries and keeps them, though formatting may be
7272
normalized. The exception is something JSON cannot rewrite exactly — a non-finite
7373
number like `1e999`, a number a rewrite would round (a very large integer, or one
74-
so small it collapses to zero), `-0`, or the same key written twice in one object
75-
— which locks the switch instead, so nothing is silently changed or dropped.
74+
so small it collapses to zero), `-0`, the same key written twice in one object, or nesting deeper
75+
than 1000 levels — which locks the switch instead, so nothing is silently changed or dropped.
7676
**OMP** is unaffected by sibling edits too, for a different reason: its writer
7777
patches only its own `providers.opencodex` range byte-wise, so the rest of the
7878
file is never rewritten. For the remaining formats that can carry comments

‎docs-site/src/content/docs/tr/guides/integrations.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,8 +85,9 @@ düzenleme — bir MCP sunucusu eklemek, kendinize ait bir sağlayıcı tanımla
8585
etrafında birleştirir ve onları korur; yalnızca biçimlendirme
8686
normalleştirilebilir. İstisna, JSON'un birebir yeniden yazamayacağı şeylerdir —
8787
`1e999` gibi sonlu olmayan bir sayı, yeniden yazımın yuvarlayacağı bir sayı (çok
88-
büyük bir tam sayı ya da sıfıra çökecek kadar küçük bir sayı), `-0` veya aynı
89-
nesnede iki kez yazılmış bir anahtar — bu durumda anahtar kilitlenir, böylece
88+
büyük bir tam sayı ya da sıfıra çökecek kadar küçük bir sayı), `-0`, aynı
89+
nesnede iki kez yazılmış bir anahtar veya 1000 seviyeden derin iç içe geçme —
90+
bu durumda anahtar kilitlenir, böylece
9091
hiçbir şey sessizce değiştirilmez veya düşürülmez. **OMP** de yanındaki
9192
düzenlemelerden etkilenmez, ama başka bir nedenle: writer'ı yalnızca kendi
9293
`providers.opencodex` aralığını bayt bayt yamalar, dosyanın geri kalanı hiçbir

‎docs-site/src/content/docs/zh-tw/guides/integrations.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ opencodex 從自己的環境讀取這些變數。如果你的 gateway 以 profil
3737
- **Restore this point…** 會出現在較舊的操作上,或當檔案在那次操作之後有變更時。跨過這樣的變更做回復會再詢問一次,才覆蓋你的較新編輯——並且也會備份它們,所以那次的回復本身也可以復原。
3838
- 每個客戶端保留十份備份。超過之後,最舊的快照檔案會被移除,其歷史列顯示為 **Backup expired**。
3939

40-
停用只移除 opencodex 記錄為自己寫入的條目。如果你的檔案在我們寫入之後有變更,後續行為取決於我們自己的條目是否完好,以及檔案的格式。對於嚴格 JSON 設定檔(OpenCode、Pi),在我們的區塊**旁邊**進行的編輯——例如新增 MCP 伺服器或你自己的 provider——會顯示為**需要更新**:重新整理會在保留你的條目的前提下合併寫入,但格式可能會被正規化。例外情況是 JSON 無法精確重寫的內容——例如 `1e999` 這類非有限數字、重寫會被四捨五入的數字(極大的整數,或小到會塌縮成零的數字)、`-0`,或同一個物件裡重複出現的鍵——此時開關會鎖定,確保沒有任何值被悄悄改動或刪除。**OMP** 同樣不受旁邊編輯影響,但原因不同:它的 writer 只逐位元組修補自己的 `providers.opencodex` 範圍,檔案其餘部分從不會被重寫。至於其餘可以包含註解的格式(Hermes、OpenClaw、Kimi Code、Gajae Code——以整份文件寫出的 YAML、JSON5 與 TOML),或當我們自己的條目被編輯過時,開關會鎖定,停用會拒絕執行,而不是猜測哪些編輯是你的。
40+
停用只移除 opencodex 記錄為自己寫入的條目。如果你的檔案在我們寫入之後有變更,後續行為取決於我們自己的條目是否完好,以及檔案的格式。對於嚴格 JSON 設定檔(OpenCode、Pi),在我們的區塊**旁邊**進行的編輯——例如新增 MCP 伺服器或你自己的 provider——會顯示為**需要更新**:重新整理會在保留你的條目的前提下合併寫入,但格式可能會被正規化。例外情況是 JSON 無法精確重寫的內容——例如 `1e999` 這類非有限數字、重寫會被四捨五入的數字(極大的整數,或小到會塌縮成零的數字)、`-0`、同一個物件裡重複出現的鍵,或巢狀層數超過 1000 層——此時開關會鎖定,確保沒有任何值被悄悄改動或刪除。**OMP** 同樣不受旁邊編輯影響,但原因不同:它的 writer 只逐位元組修補自己的 `providers.opencodex` 範圍,檔案其餘部分從不會被重寫。至於其餘可以包含註解的格式(Hermes、OpenClaw、Kimi Code、Gajae Code——以整份文件寫出的 YAML、JSON5 與 TOML),或當我們自己的條目被編輯過時,開關會鎖定,停用會拒絕執行,而不是猜測哪些編輯是你的。
4141

4242
## 誠實的預期
4343

‎src/integrations/config-io.ts‎

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
*/
1010
import { mkdirSync, readFileSync, rmSync, statSync } from "node:fs";
1111
import type { ConfigFormat } from "../clients/config-export";
12+
import { MAX_JSON_NESTING } from "./serialize";
1213
import { atomicWriteFile } from "../config";
1314
import type { JournalEntry } from "./journal";
1415
import type { OwnershipRecord } from "./ownership";
@@ -42,10 +43,21 @@ export const PARSE_FAILED = Symbol("parse-failed");
4243
* normalization we promise, and this classifier is what makes the rewrite of a
4344
* user-edited file reachable at all — so it fails closed here.
4445
*
45-
* Scanning also avoids recursing over attacker-shaped nesting depth.
46+
* Depth: the same pass counts container nesting against MAX_JSON_NESTING
47+
* (shared with the serializer, see serialize.ts). JSON.parse handles hundreds
48+
* of thousands of levels iteratively, but the downstream merge and
49+
* JSON.stringify recurse — a 100KB file nested 50k deep sailed through parse
50+
* and guard, then blew up serialization with a raw RangeError after a
51+
* multi-GB allocation spike. A hand-written scan rather than a JSON.parse
52+
* source-access reviver on purpose: the reviver walk recurses internally, so
53+
* its depth limit would be an unpredictable stack size instead of this
54+
* deterministic ceiling.
4655
*/
4756
function jsonTextSafeToRewrite(text: string): boolean {
48-
/** One frame per open container; a Set for objects, null for arrays. */
57+
/**
58+
* One frame per open container; a Set of member names for objects, null for
59+
* arrays. Its length is the current nesting depth.
60+
*/
4961
const containers: Array<Set<string> | null> = [];
5062
/** The most recent string literal — the member name if a `:` follows. */
5163
let lastString: string | null = null;
@@ -66,6 +78,7 @@ function jsonTextSafeToRewrite(text: string): boolean {
6678
if (ch === "\"") { inString = true; stringStart = i; continue; }
6779
if (ch === "{" || ch === "[") {
6880
containers.push(ch === "{" ? new Set<string>() : null);
81+
if (containers.length > MAX_JSON_NESTING) return false;
6982
lastString = null;
7083
continue;
7184
}
@@ -124,8 +137,15 @@ export function parseConfig(text: string | null, format: ConfigFormat): unknown
124137
try {
125138
switch (format) {
126139
case "json": {
127-
const parsed = JSON.parse(text);
128-
return jsonTextSafeToRewrite(text) ? parsed : PARSE_FAILED;
140+
/*
141+
* Scanned BEFORE parsing: the guard is text-only, and refusing first
142+
* means a hostile document is never materialized — the depth ceiling
143+
* would otherwise cap the rewrite only after JSON.parse had already
144+
* built the 50k-deep object graph. The outcome is unchanged: invalid
145+
* JSON still returns PARSE_FAILED, from the catch below.
146+
*/
147+
if (!jsonTextSafeToRewrite(text)) return PARSE_FAILED;
148+
return JSON.parse(text);
129149
}
130150
case "json5": return Bun.JSON5.parse(text);
131151
case "yaml": return Bun.YAML.parse(text);

‎src/integrations/serialize.ts‎

Lines changed: 80 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -221,10 +221,89 @@ export function renderToml(document: Record<string, unknown>, prefix = ""): stri
221221
return `${[scalars.join("\n"), tables.join("\n\n")].filter(Boolean).join("\n\n")}\n`;
222222
}
223223

224+
/**
225+
* Ceiling on container nesting for json documents, shared by the parse-time
226+
* scanner (config-io.ts) and the serializer walk below. One constant on
227+
* purpose: the walk must accept every document the scanner admits, or a file
228+
* the classifier reported as recoverable would refuse at rewrite time. Real
229+
* configs nest a handful of levels.
230+
*/
231+
export const MAX_JSON_NESTING = 1000;
232+
233+
/** Error messages carry the path to the offending value; keep them readable. */
234+
function clampPath(path: string): string {
235+
return path.length > 200 ? `${path.slice(0, 100)}…${path.slice(-100)}` : path;
236+
}
237+
238+
/**
239+
* JSON.stringify writes a non-finite number as `null` and -0 as `0`; any
240+
* other finite double round-trips value-exactly (literal-level rounding is
241+
* the parse-time scanner's concern), so those two are exactly what this walk
242+
* refuses — refusing more turned a state the classifier had promised as
243+
* recoverable into a permanent refusal. Documents read from disk are already
244+
* guarded at parse time, and the writer's merge layer JSON-clones documents —
245+
* normalizing these values — before serializing, so on the apply/disable path
246+
* this walk is unreachable for them: it guards the direct serializers
247+
* (preview/export builders), same posture as the YAML and TOML renderers
248+
* above, and enforces the nesting ceiling for every json caller before the
249+
* recursive JSON.stringify can turn depth into a RangeError.
250+
*
251+
* Iterative frames instead of recursion or a node stack: depth AND size of
252+
* the document are inputs under the writer of the config file. Recursion made
253+
* a deep file a RangeError-500; materializing every node with its path made a
254+
* wide file allocate a large multiple of its size. Frames keep memory
255+
* proportional to nesting depth, and path strings exist only for the
256+
* containers on the current path plus the failing value itself.
257+
*/
258+
function assertJsonNumbersRoundTrip(document: unknown, rootPath: string): void {
259+
const refuse = (value: number, path: string): never => {
260+
throw new UnserializableValueError(Object.is(value, -0)
261+
? `JSON cannot rewrite -0 at ${clampPath(path)} without changing it to 0`
262+
: `JSON cannot rewrite the number at ${clampPath(path)} without changing it to null`);
263+
};
264+
if (typeof document === "number" && (!Number.isFinite(document) || Object.is(document, -0))) {
265+
refuse(document, rootPath);
266+
}
267+
type Frame = { container: unknown; keys: string[] | null; index: number; prefix: string };
268+
const frames: Frame[] = [];
269+
const pushContainer = (value: unknown, prefix: string) => {
270+
if (Array.isArray(value)) frames.push({ container: value, keys: null, index: 0, prefix });
271+
else if (isPlainRecord(value)) frames.push({ container: value, keys: Object.keys(value), index: 0, prefix });
272+
};
273+
pushContainer(document, rootPath);
274+
while (frames.length > 0) {
275+
const frame = frames[frames.length - 1]!;
276+
const length = frame.keys ? frame.keys.length : (frame.container as unknown[]).length;
277+
if (frame.index >= length) { frames.pop(); continue; }
278+
const i = frame.index;
279+
frame.index += 1;
280+
const child = frame.keys
281+
? (frame.container as Record<string, unknown>)[frame.keys[i]!]
282+
: (frame.container as unknown[])[i];
283+
const childPath = () => frame.keys
284+
? (frame.prefix === "$" ? frame.keys[i]! : `${frame.prefix}.${frame.keys[i]!}`)
285+
: `${frame.prefix}[${i}]`;
286+
if (typeof child === "number") {
287+
if (!Number.isFinite(child) || Object.is(child, -0)) refuse(child, childPath());
288+
continue;
289+
}
290+
if (typeof child === "object" && child !== null) {
291+
if (frames.length >= MAX_JSON_NESTING) {
292+
throw new UnserializableValueError(
293+
`the document nests deeper than ${MAX_JSON_NESTING} levels at ${clampPath(childPath())}, which JSON serialization cannot rewrite safely`);
294+
}
295+
pushContainer(child, childPath());
296+
}
297+
}
298+
}
299+
224300
/** Every serializer returns text ending in exactly one newline. */
225301
export function serializeDocument(document: unknown, format: ConfigFormat): string {
226302
switch (format) {
227-
case "json": return `${JSON.stringify(document, null, 2)}\n`;
303+
case "json": {
304+
assertJsonNumbersRoundTrip(document, "$");
305+
return `${JSON.stringify(document, null, 2)}\n`;
306+
}
228307
case "json5": return `${Bun.JSON5.stringify(document, null, 2)}\n`;
229308
case "yaml": return renderYaml(document);
230309
case "toml": {

‎src/integrations/writer.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ function preflight(input: IntegrationWriteInput) {
214214
const parsed = parseConfig(before, exportSpec.format);
215215
if (parsed === PARSE_FAILED) {
216216
return { failed: refuse(clientId, "unsafe", "unsafe",
217-
`${configPath} could not be parsed, or holds something opencodex cannot rewrite without changing it (a non-finite number, a large integer or a tiny one a rewrite would round, -0, or a duplicate member)`) } as const;
217+
`${configPath} could not be parsed, or holds something opencodex cannot rewrite without changing it (a non-finite number, a large integer or a tiny one a rewrite would round, -0, a duplicate member, or nesting deeper than 1000 levels)`) } as const;
218218
}
219219
const contribution = exportSpec.buildContribution(exportContextOf(input));
220220
// A record proves ownership of the file it was written FOR. Matching only by

‎tests/integrations-serialize.test.ts‎

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { describe, expect, test } from "bun:test";
22
import {
33
FORMAT_MEDIA_TYPE,
4+
UnserializableValueError,
45
quoteTomlKey,
56
renderToml,
67
renderYaml,
@@ -155,9 +156,56 @@ describe("serializeDocument", () => {
155156
expect(() => serializeDocument([1, 2], "toml")).toThrow(/TOML root must be a table/);
156157
});
157158

159+
test("json refuses numbers that would not survive the rewrite", () => {
160+
expect(() => serializeDocument({ q: Infinity }, "json")).toThrow(UnserializableValueError);
161+
expect(() => serializeDocument({ q: Number.NaN }, "json")).toThrow(UnserializableValueError);
162+
expect(() => serializeDocument({ q: -0 }, "json")).toThrow(UnserializableValueError);
163+
// The failure names where the value sits — including through arrays and
164+
// for a bare root scalar.
165+
expect(() => serializeDocument({ a: { b: [1, -Infinity] } }, "json")).toThrow(/a\.b\[1\]/);
166+
expect(() => serializeDocument({ a: [{ b: -0 }] }, "json")).toThrow(/a\[0\]\.b/);
167+
expect(() => serializeDocument(-0, "json")).toThrow(/at \$/);
168+
// Empty containers pass the walk untouched.
169+
expect(serializeDocument({}, "json")).toBe("{}\n");
170+
expect(serializeDocument([], "json")).toBe("[]\n");
171+
});
172+
173+
test("json keeps every finite non-negative-zero double, however large", () => {
174+
// A finite double round-trips value-exactly; refusing 2^54 here while the
175+
// parse-time scanner admits it turned a classifier-promised 'stale' into
176+
// a permanent 'unsafe' refusal.
177+
expect(JSON.parse(serializeDocument({ q: 2 ** 54 }, "json"))).toEqual({ q: 2 ** 54 });
178+
expect(JSON.parse(serializeDocument({ q: 1e21 }, "json"))).toEqual({ q: 1e21 });
179+
});
180+
181+
test("hostile nesting depth is a structured refusal, not a RangeError", () => {
182+
// The walk itself is iterative, and past MAX_SERIALIZED_JSON_NESTING it
183+
// refuses — JSON.stringify recurses, so letting a 50k-deep document
184+
// through would trade the refusal for a multi-GB spike and a raw
185+
// RangeError. The refusal message stays readable (clamped path).
186+
let deep: unknown = -Infinity;
187+
for (let i = 0; i < 100_000; i += 1) deep = [deep];
188+
try {
189+
serializeDocument({ q: deep }, "json");
190+
throw new Error("expected a refusal");
191+
} catch (error) {
192+
expect(error).toBeInstanceOf(UnserializableValueError);
193+
expect((error as Error).message).toContain("nests deeper");
194+
expect((error as Error).message.length).toBeLessThan(500);
195+
// The clamp keeps head and tail of the path, joined by an ellipsis.
196+
expect((error as Error).message).toMatch(/q\[0\].*….*\[0\]/);
197+
}
198+
});
158199

159-
160-
200+
test("nesting within the ceiling still serializes, up to the exact boundary", () => {
201+
// The ceiling admits exactly MAX_JSON_NESTING container levels — the same
202+
// count the parse-time scanner admits. Pinning both edges catches an
203+
// off-by-one drift in either layer.
204+
let atCeiling: unknown = 1;
205+
for (let i = 0; i < 1000; i += 1) atCeiling = [atCeiling];
206+
expect(() => serializeDocument(atCeiling, "json")).not.toThrow();
207+
expect(() => serializeDocument([atCeiling], "json")).toThrow(/nests deeper/);
208+
});
161209

162210
test("media types are declared for every format", () => {
163211
expect(Object.keys(FORMAT_MEDIA_TYPE).sort()).toEqual(["json", "json5", "toml", "yaml"]);

‎tests/integrations-state.test.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";
44
import { join } from "node:path";
55
import { EXPORT_CLIENTS, type ExportModel } from "../src/clients/config-export";
66
import { PARSE_FAILED, fileIO, loadTarget, parseConfig } from "../src/integrations/config-io";
7+
import { serializeDocument } from "../src/integrations/serialize";
78
import {
89
canonicalContribution,
910
fingerprint,
@@ -458,6 +459,21 @@ describe("classifier unit behavior", () => {
458459
expect(parseConfig("{\"a\": 1e-320}", "json")).toEqual({ a: 1e-320 });
459460
});
460461

462+
test("parseConfig refuses json nested deeper than the rewrite can carry", () => {
463+
// Nesting past the ceiling: parse would succeed, but the downstream
464+
// rewrite machinery recurses — refuse at the trust boundary.
465+
expect(parseConfig(`${"[".repeat(1001)}1${"]".repeat(1001)}`, "json")).toBe(PARSE_FAILED);
466+
// The exact boundary: what the scanner admits, the serializer must also
467+
// rewrite — one document through both layers, or a config the classifier
468+
// reported recoverable would refuse at rewrite time.
469+
const atCeiling = parseConfig(`${"[".repeat(1000)}1${"]".repeat(1000)}`, "json");
470+
expect(atCeiling).not.toBe(PARSE_FAILED);
471+
expect(() => serializeDocument(atCeiling, "json")).not.toThrow();
472+
// Brackets inside strings do not count toward depth.
473+
expect(parseConfig(`{"a": "${"[".repeat(2000)}"}`, "json"))
474+
.toEqual({ a: "[".repeat(2000) });
475+
});
476+
461477
test("parseConfig keeps json numbers that round-trip exactly", () => {
462478
// 1e21 and 2^54 are exactly representable doubles; only the literal's
463479
// spelling may normalize, never the value any JSON consumer reads.

0 commit comments

Comments
 (0)