Repository navigation
Grep tool fails with "invalid ripgrep output" when source files contain non-UTF-8 (e.g. GBK) characters #23629
Description
Description
Bug description
The built-in Grep tool always fails with the error "invalid ripgrep output" when any matched file contains non-UTF-8 encoded characters (e.g. GBK/GB2312 Chinese comments or text files).
Steps to reproduce
- Have a project that contains source files with GBK-encoded content
- Use the Grep tool to search for any pattern that matches those files
- Grep tool returns: invalid ripgrep output
Root cause
When ripgrep encounters bytes that cannot be decoded as valid UTF-8, it outputs the matched line as lines.bytes (base64-encoded) instead of lines.text in its JSON format.
Example of the JSON ripgrep actually emits:
{"type":"match","data":{"path":{"text":"./server/sctrl/main.cpp"},
"lines":{"bytes":"CS8vRU5BQkxFX0VYU0RL..."}, ← bytes, not text!
"line_number":1780, ...}}
opencode's ripgrep JSON parser only handles lines.text and throws "invalid ripgrep output" when it encounters lines.bytes, even though the rest of the results (all UTF-8 files) are perfectly valid.
This means a single non-UTF-8 file in the project completely breaks the Grep tool for ALL searches, regardless of whether the specific search matches that file.
Expected behavior
The Grep tool should handle lines.bytes gracefully — either by base64-decoding it, replacing it with a placeholder like [binary/non-UTF-8 line], or skipping only that specific match while returning all valid UTF-8 matches.
Workaround
Use the Bash tool with rg --text instead:
rg --text "PATTERN" .
The --text flag forces ripgrep to treat all files as text, avoiding the bytes field entirely.
Environment
- opencode version: latest
- OS: Ubuntu Linux (x86_64)
- ripgrep version: 15.1.0
- Affected file types: GBK-encoded .c / .cpp / .txt files
Plugins
No response
OpenCode version
No response
Steps to reproduce
No response
Screenshot and/or share link
No response
Operating System
ubuntu18
Terminal
No response
Activity
kagura-chen commentedon Apr 21, 2026
I'd like to work on this. The root cause is in
src/file/ripgrep.ts— theMatchzod schema only acceptslines.textbut ripgrep emitslines.bytes(base64) for non-UTF-8 content, causing theparse()function to throw. Same issue applies topath.bytesandsubmatches[].match.bytes.Plan: update the schemas to accept either
textorbytes, and base64-decode the bytes variant with a lossy UTF-8 fallback.fix(grep): handle non-UTF-8 ripgrep output (lines.bytes)
SuperStrain commentedon May 19, 2026
Could this be related to this issue?
#27832
kagura-chen commentedon May 19, 2026
Good catch @SuperStrain — #27832 is definitely related! Both stem from the same root cause: OpenCode's built-in tools assume UTF-8 encoding. #27832 proposes broader multi-encoding support across Read/Grep/Edit/Write, while this issue specifically targets the ripgrep output parser crash when
lines.bytes(base64) is returned instead oflines.text.This issue's fix (handling the
bytesvariant in the Zod schema) would be a targeted first step that also benefits the broader #27832 effort. The parser should gracefully handle non-UTF-8 content even before full multi-encoding support lands.SuperStrain commentedon May 26, 2026
I hope this issue can receive some feedback. Thank you. @jlongster
jasonr666 commentedon May 27, 2026
I'm encountering the same problem and hope it can be resolved soon. Thank you.
github-actions commentedon Jul 27, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.