Skip to content

Commit 32cfa33

Browse files
sahrizviclaude
andcommitted
fix(core): reject unaddressable submatch offsets instead of clamping
Second bot-review round (cubic, kilo). - `Buffer.subarray` clamps an out-of-range end and truncates a fractional one rather than throwing, so rebasing an offset without a range check quietly repaired a corrupt offset into a plausible-looking one. Neither schema catches it: core `NonNegativeInt` and legacy `z.number()` both accept a number well past the end of the line. An unaddressable offset now marks the record corrupt so it is skipped and counted, in both parsers. - Correct an overstated comment: the win32 skip claimed Windows ripgrep behaviour was covered by script/windows-ripgrep-e2e.ts, but that script covers only binary resolution, extraction and one real search — none of the record-parsing behaviour these stub cases pin. The comment now states the gap. Not changed, with reasons: - Submatch offsets still index the full line after the 2000-char cap. That is the tracked windowing follow-up, and the observable output is unchanged by this branch — the cap moved earlier, it did not become lossier. - The legacy parser still decodes every submatch rather than slicing to MAX_SUBMATCHES first. Its response shape is published by `/find`, so slicing would change that contract; the cost is already bounded by the record ceiling. - The second capLineText call in the result mapping is a deliberate guard on the public output, not dead code, and is documented as such. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1 parent fe122a9 commit 32cfa33

4 files changed

Lines changed: 68 additions & 11 deletions

File tree

‎packages/core/src/ripgrep.ts‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -130,12 +130,24 @@ const normalizeMatch = (json: object): unknown => {
130130
const lines = decodeField(readProp(data, "lines"))
131131
if (!lines) return json
132132
const raw = lines.raw
133-
const rebase = (offset: unknown): unknown =>
134-
raw && typeof offset === "number" && offset >= 0
135-
? Buffer.byteLength(raw.subarray(0, offset).toString("utf8"), "utf8")
136-
: offset
133+
// Rebasing is a claim about coordinates, so it is only made for an offset actually addressable in
134+
// the raw line. `Buffer.subarray` clamps an out-of-range end and truncates a fractional one rather
135+
// than throwing, so an unchecked rebase would quietly repair a corrupt offset into a
136+
// plausible-looking one — and the schema would not catch it, since `NonNegativeInt` happily
137+
// accepts a number past the end of the line. An unaddressable offset therefore marks the whole
138+
// record corrupt (undefined) so it is skipped and counted. Offsets on the `{text}` arm are left
139+
// alone: no rebasing happens there, so no claim is made and ripgrep's values stand as before.
140+
let corrupt = false
141+
const rebase = (offset: unknown): unknown => {
142+
if (!raw) return offset
143+
if (typeof offset !== "number" || !Number.isInteger(offset) || offset < 0 || offset > raw.length) {
144+
corrupt = true
145+
return offset
146+
}
147+
return Buffer.byteLength(raw.subarray(0, offset).toString("utf8"), "utf8")
148+
}
137149
const submatches = readProp(data, "submatches")
138-
return {
150+
const normalized = {
139151
...json,
140152
data: {
141153
...data,
@@ -158,6 +170,7 @@ const normalizeMatch = (json: object): unknown => {
158170
: submatches,
159171
},
160172
}
173+
return corrupt ? undefined : normalized
161174
}
162175
// altimate_change end
163176

‎packages/core/test/ripgrep.test.ts‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,11 @@ describe("Ripgrep", () => {
189189
])
190190

191191
// The stub is a POSIX shell script and `chmod` is a no-op on win32, so the spawn cannot work
192-
// there. Windows ripgrep behaviour has its own coverage in script/windows-ripgrep-e2e.ts.
192+
// there. Be clear about the cost: this leaves the record-parsing behaviour below — skipping,
193+
// decoding, offset rebasing, the size ceiling — WITHOUT Windows coverage. script/windows-
194+
// ripgrep-e2e.ts covers only binary resolution, extraction and one real search, not any of this.
195+
// Closing the gap needs a `.cmd` stub on win32; the parser itself is platform-independent, so the
196+
// risk is a Windows-only spawn/quoting regression going unnoticed rather than a parsing one.
193197
const stubTest = bunTest.skipIf(process.platform === "win32")
194198

195199
const grepWithStubbedRecords = (records: string[]) =>
@@ -306,6 +310,26 @@ describe("Ripgrep", () => {
306310
expect(Buffer.from(text, "utf8").subarray(submatches[0].start, submatches[0].end).toString("utf8")).toBe("needle")
307311
})
308312

313+
// `Buffer.subarray` clamps an out-of-range end and truncates a fractional one instead of throwing,
314+
// so rebasing without a range check would turn a corrupt offset into a plausible-looking one.
315+
stubTest("skips a record whose submatch offset is not addressable in the line", async () => {
316+
const raw = Buffer.concat([Buffer.from([0xff]), Buffer.from("needle tail\n")])
317+
const matches = await Effect.runPromise(
318+
grepWithStubbedRecords([
319+
matchRecord("a.txt"),
320+
matchRecord("b.txt", {
321+
lines: { bytes: raw.toString("base64") },
322+
// Well past the end of the raw line — nonsense that must not be silently clamped.
323+
submatches: [{ match: { text: "needle" }, start: 1, end: 9_999 }],
324+
}),
325+
matchRecord("c.txt"),
326+
]),
327+
)
328+
329+
expect(matches.map((item) => item.entry.path)).toEqual([RelativePath.make("a.txt"), RelativePath.make("c.txt")])
330+
expect(lastSkip()?.skipped).toBe(1)
331+
})
332+
309333
stubTest("skips a record whose bytes field is empty rather than emitting an empty match", async () => {
310334
const matches = await Effect.runPromise(
311335
grepWithStubbedRecords([matchRecord("a.txt"), matchRecord("b.txt", { lines: { bytes: "" } })]),

‎packages/opencode/src/file/ripgrep.ts‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -141,17 +141,26 @@ export namespace Ripgrep {
141141
// `/find` route, so unrebased offsets would be newly wrong output rather than a skipped record.
142142
// Mirrors packages/core/src/ripgrep.ts.
143143
const raw = lines?.raw
144-
const rebase = (offset: unknown): unknown =>
145-
raw && typeof offset === "number" && offset >= 0
146-
? Buffer.byteLength(raw.subarray(0, offset).toString("utf8"), "utf8")
147-
: offset
144+
// Only rebase an offset addressable in the raw line: `Buffer.subarray` clamps an out-of-range
145+
// end and truncates a fractional one rather than throwing, so an unchecked rebase would quietly
146+
// repair a corrupt offset, and the schema would not catch it (`z.number()` accepts any number).
147+
// An unaddressable offset marks the record corrupt so it is skipped and counted instead.
148+
let corrupt = false
149+
const rebase = (offset: unknown): unknown => {
150+
if (!raw) return offset
151+
if (typeof offset !== "number" || !Number.isInteger(offset) || offset < 0 || offset > raw.length) {
152+
corrupt = true
153+
return offset
154+
}
155+
return Buffer.byteLength(raw.subarray(0, offset).toString("utf8"), "utf8")
156+
}
148157
const submatches = read(data, "submatches")
149158
// Only rewrite keys the record actually carries — `begin`/`end`/`summary` records reach here too
150159
// and must keep their exact shape, or the strict union below would reject them.
151160
// `path` is deliberately left alone: decoding it is lossy, and a path is an identifier the
152161
// caller reopens, so a U+FFFD-mangled path names a file that does not exist. Such a record
153162
// stays in the `{bytes}` arm and is skipped. See packages/core/src/ripgrep.ts.
154-
return {
163+
const normalized = {
155164
...json,
156165
data: {
157166
...data,
@@ -173,6 +182,7 @@ export namespace Ripgrep {
173182
: {}),
174183
},
175184
}
185+
return corrupt ? undefined : normalized
176186
}
177187

178188
/**

‎packages/opencode/test/file/ripgrep-search.test.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,16 @@ describe("legacy Ripgrep.parseRecords", () => {
113113
expect(Buffer.from(lines.text, "utf8").subarray(3, 9).toString("utf8")).toBe("needle")
114114
})
115115

116+
test("skips a record whose submatch offset is not addressable in the line", () => {
117+
const raw = Buffer.concat([Buffer.from([0xff]), Buffer.from("needle tail\n")])
118+
const bad = record("b.txt", {
119+
lines: { bytes: raw.toString("base64") },
120+
submatches: [{ match: { text: "needle" }, start: 1, end: 9_999 }],
121+
})
122+
// `z.number()` would accept 9999 happily, so the range check is what rejects this.
123+
expect(paths([record("a.txt"), bad, record("c.txt")])).toEqual(["./a.txt", "./c.txt"])
124+
})
125+
116126
test("returns an empty array when every record is unusable, rather than throwing", () => {
117127
expect(() => Ripgrep.parseRecords(["{oops", "{also oops"])).not.toThrow()
118128
expect(Ripgrep.parseRecords(["{oops", "{also oops"])).toEqual([])

0 commit comments

Comments
 (0)