Skip to content

Commit be350ee

Browse files
committed
fix(grep): handle non-UTF-8 ripgrep output (lines.bytes)
When ripgrep encounters non-UTF-8 encoded files (e.g. GBK/GB2312), it emits lines.bytes (base64) instead of lines.text. The zod schema only accepted lines.text, causing parse failures that broke grep for all searches in projects containing any non-UTF-8 files. Add a TextOrBytes schema that accepts either text or bytes fields, base64-decoding the bytes variant with lossy UTF-8 conversion. Applied to both lines and submatches.match fields. Fixes anomalyco#23629
1 parent a38d53f commit be350ee

1 file changed

Lines changed: 12 additions & 6 deletions

File tree

‎packages/opencode/src/file/ripgrep.ts‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import { Buffer } from "buffer"
12
import path from "path"
23
import z from "zod"
34
import { AppFileSystem } from "@opencode-ai/shared/filesystem"
@@ -48,22 +49,27 @@ const Begin = z.object({
4849
}),
4950
})
5051

52+
const TextOrBytes = z
53+
.object({
54+
text: z.string().optional(),
55+
bytes: z.string().optional(),
56+
})
57+
.transform((val) => ({
58+
text: val.text ?? Buffer.from(val.bytes ?? "", "base64").toString("utf-8"),
59+
}))
60+
5161
export const Match = z.object({
5262
type: z.literal("match"),
5363
data: z.object({
5464
path: z.object({
5565
text: z.string(),
5666
}),
57-
lines: z.object({
58-
text: z.string(),
59-
}),
67+
lines: TextOrBytes,
6068
line_number: z.number(),
6169
absolute_offset: z.number(),
6270
submatches: z.array(
6371
z.object({
64-
match: z.object({
65-
text: z.string(),
66-
}),
72+
match: TextOrBytes,
6773
start: z.number(),
6874
end: z.number(),
6975
}),

0 commit comments

Comments
 (0)