fix: set default UTF-8 encoding for spawned subprocess on Windows by Thalynor · Pull Request #31658 · anomalyco/opencode

Thalynor · 2026-06-10T10:11:43Z

Issue for this PR

Closes #23636
Closes #31187
Closes #30205
Closes #31830
Closes #26882

Type of change

Bug fix
New feature
Refactor / code improvement
Documentation

What does this PR do?

On zh-CN Windows systems, the default active code page (ACP) is 936 (GBK). New subprocesses spawned by OpenCode inherit this encoding. Garbled characters occur when tools output non-ASCII characters such as CJK characters.

This PR fixes the garbled text bug by prepending the command $OutputEncoding = [Console]::OutputEncoding = [System.Text.UTF8Encoding]::new($false); before the executed command to set the character encoding to UTF-8; this fix targets PowerShell. When the spawned subprocess is CMD, chcp 65001 >nul is prepended to the command to switch the code page.

When running the python command in Windows Terminal, if stdout is a pipe (rather than the console), Python ignores the code page set via chcp and falls back to the system ANSI code page (GBK on Chinese-language Windows systems). The fix is to append the environment variable PYTHONIOENCODING=utf-8 in the shell environment.

How did you verify your code works?

I ran OpenCode locally with bun dev. In shell mode, I ran the ls command to list a directory containing files with Chinese filenames, and the output displayed correctly. I also executed Write-Output "中文输出" to inspect console output, which rendered properly as well.

I have completed local tests with both PowerShell and CMD set as the default shells. The results meet expectations with no garbled characters present.

Screenshots / recordings

before

after

Checklist

I have tested my changes locally
I have not included unrelated changes in this PR

On non-UTF-8 Windows locales (e.g. zh-CN ACP 936/GBK), shell subprocesses emit output in the legacy code page which we decode as UTF-8, producing mojibake for CJK filenames/output. Fix both execution paths: - bash tool (src/tool/bash.ts): prepend an $OutputEncoding/[Console]:: OutputEncoding UTF-8 statement for PowerShell, chcp 65001 for cmd. - TUI shell mode (src/session/prompt.ts shellImpl): same prefixes for powershell/pwsh/cmd invocations. - Add PYTHONIOENCODING=utf-8 to the shell env on Windows so piped Python output is UTF-8 (Python ignores the code page when stdout is a pipe). Based on upstream anomalyco/opencode#31658.

…1418) * fix: force UTF-8 for Windows shell subprocesses to fix CJK mojibake On non-UTF-8 Windows locales (e.g. zh-CN ACP 936/GBK), shell subprocesses emit output in the legacy code page which we decode as UTF-8, producing mojibake for CJK filenames/output. Fix both execution paths: - bash tool (src/tool/bash.ts): prepend an $OutputEncoding/[Console]:: OutputEncoding UTF-8 statement for PowerShell, chcp 65001 for cmd. - TUI shell mode (src/session/prompt.ts shellImpl): same prefixes for powershell/pwsh/cmd invocations. - Add PYTHONIOENCODING=utf-8 to the shell env on Windows so piped Python output is UTF-8 (Python ignores the code page when stdout is a pipe). Based on upstream anomalyco/opencode#31658. * docs: add Windows CJK mojibake workaround to README Document the system-wide UTF-8 (Beta) toggle as a workaround for garbled CJK shell output on non-UTF-8 Windows locales, for users on older versions or tools not yet special-cased. Added to both English and Chinese README. * docs: clarify MiMoCode version and tone in Windows CJK note * docs: refine wording of Windows CJK note * refactor: extract shared Windows UTF-8 shell prefixes; redirect chcp stderr Address PR review: extract POWERSHELL_UTF8_PREFIX and CMD_UTF8_PREFIX into src/shell/shell.ts so the bash tool and TUI shell-mode paths can't drift, and redirect both stdout and stderr (>nul 2>nul) so a chcp failure in a restricted shell never pollutes command output.

…iaomiMiMo#1418) * fix: force UTF-8 for Windows shell subprocesses to fix CJK mojibake On non-UTF-8 Windows locales (e.g. zh-CN ACP 936/GBK), shell subprocesses emit output in the legacy code page which we decode as UTF-8, producing mojibake for CJK filenames/output. Fix both execution paths: - bash tool (src/tool/bash.ts): prepend an $OutputEncoding/[Console]:: OutputEncoding UTF-8 statement for PowerShell, chcp 65001 for cmd. - TUI shell mode (src/session/prompt.ts shellImpl): same prefixes for powershell/pwsh/cmd invocations. - Add PYTHONIOENCODING=utf-8 to the shell env on Windows so piped Python output is UTF-8 (Python ignores the code page when stdout is a pipe). Based on upstream anomalyco/opencode#31658. * docs: add Windows CJK mojibake workaround to README Document the system-wide UTF-8 (Beta) toggle as a workaround for garbled CJK shell output on non-UTF-8 Windows locales, for users on older versions or tools not yet special-cased. Added to both English and Chinese README. * docs: clarify MiMoCode version and tone in Windows CJK note * docs: refine wording of Windows CJK note * refactor: extract shared Windows UTF-8 shell prefixes; redirect chcp stderr Address PR review: extract POWERSHELL_UTF8_PREFIX and CMD_UTF8_PREFIX into src/shell/shell.ts so the bash tool and TUI shell-mode paths can't drift, and redirect both stdout and stderr (>nul 2>nul) so a chcp failure in a restricted shell never pollutes command output.

…1418) * fix: force UTF-8 for Windows shell subprocesses to fix CJK mojibake On non-UTF-8 Windows locales (e.g. zh-CN ACP 936/GBK), shell subprocesses emit output in the legacy code page which we decode as UTF-8, producing mojibake for CJK filenames/output. Fix both execution paths: - bash tool (src/tool/bash.ts): prepend an $OutputEncoding/[Console]:: OutputEncoding UTF-8 statement for PowerShell, chcp 65001 for cmd. - TUI shell mode (src/session/prompt.ts shellImpl): same prefixes for powershell/pwsh/cmd invocations. - Add PYTHONIOENCODING=utf-8 to the shell env on Windows so piped Python output is UTF-8 (Python ignores the code page when stdout is a pipe). Based on upstream anomalyco/opencode#31658. * docs: add Windows CJK mojibake workaround to README Document the system-wide UTF-8 (Beta) toggle as a workaround for garbled CJK shell output on non-UTF-8 Windows locales, for users on older versions or tools not yet special-cased. Added to both English and Chinese README. * docs: clarify MiMoCode version and tone in Windows CJK note * docs: refine wording of Windows CJK note * refactor: extract shared Windows UTF-8 shell prefixes; redirect chcp stderr Address PR review: extract POWERSHELL_UTF8_PREFIX and CMD_UTF8_PREFIX into src/shell/shell.ts so the bash tool and TUI shell-mode paths can't drift, and redirect both stdout and stderr (>nul 2>nul) so a chcp failure in a restricted shell never pollutes command output.

Leon0714 · 2026-08-12T01:14:51Z

为什么还没被合并。。。。非常需要这个

Bigzhangbig · 2026-09-29T14:51:23Z

Still broken on v2.0.18 (windows-x64) — verified today.

Two notes for the rebase:

PS7 needs the injection too. On zh-CN, pwsh 7.6.6 reports:

# stdout piped (how opencode runs it)
PS> [Console]::OutputEncoding.CodePage   # 936
PS> [Console]::OutputEncoding.WebName    # gb2312

# fresh conhost window, no redirection
936|gb2312

GetConsoleOutputCP() returns the NLS OEMCP, so the parent process doesn't matter. Any version check that skips the preamble for PS7 will leave zh-CN / ja-JP / ko-KR broken.

The patch site moved. On v2.0.18 the arg builder is a single fU() helper shared by the bash tool and the TUI, so the cmd() branch shown in the description no longer exists as a separate function.

The three-layer fix in the description is correct — all of chcp 65001, [Console]::OutputEncoding, $OutputEncoding are required.

Written with opencode v2.0.18; all numbers measured on my own machine.

xiaozhu1337 · 2026-10-03T05:01:37Z

Still reproduces on v2.0.22 (windows-x64) — independently verified today.

Environment

OS: Windows NT 10.0.26300.0 (zh-CN); registry ACP/OEMCP = 936
shell: pwsh 7.6.6 (C:\Program Files\WindowsApps\Microsoft.PowerShell_7.6.6.0_x64__8wekyb3d8bbwe\pwsh.EXE)
opencode: v2.0.22

Repro / measurement (as seen by the spawned child with piped stdout)

[Console]::OutputEncoding.CodePage  -> 936
[Console]::OutputEncoding.WebName   -> gb2312
Write-Output '中文测试你好世界'      -> ���Ĳ����������   (mojibake)
Write-Output 'abc123'               -> abc123             (ASCII unaffected)

Two notes for the rebase:

PS7 must be injected too. On zh-CN, pwsh 7.6.6 spawned by opencode with piped stdout still reports [Console]::OutputEncoding = 936 / gb2312. Any version check that skips the preamble for PS7 leaves zh-CN / ja-JP / ko-KR broken. (Independent confirmation of @Bigzhangbig's measurement.)
The preamble must be prepended, not appended. .NET captures the stdout encoder on the first write, so setting [Console]::OutputEncoding after an earlier Write-Output in the same command does not fix that output. Prepending is correct today — flagging it so a rebase does not move it.

Confirmed the three-layer prefix works (chcp 65001 for cmd; [Console]::OutputEncoding + [Console]::InputEncoding + $OutputEncoding for PowerShell; PYTHONIOENCODING=utf-8 for piped Python).

As a data point that the mechanism is viable on v2.0.22 without a core patch: the V2 plugin [email protected] hooks shell create.before and injects exactly this prefix. With it, Write-Output '中文编码测试ABC' returns 中文编码测试ABC correctly, and the spawned shell has PYTHONIOENCODING=utf-8.

Hoping this gets merged — the duplicate issues keep multiplying.

thdxr · 2026-10-10T17:37:12Z

This branch has been upgraded to V2. We're no longer taking PRs for V1. If you think this is still relevant, please port it over and open it for V2.

— from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

fix: default UTF-8 encoding for spawned subprocess on Windows

533d91d

github-actions Bot mentioned this pull request Jun 10, 2026

📊 AI CLI 工具社区动态日报 2026-06-10 hehongtao88/big_model_radar#6

Open

fix: force UTF-8 for Windows shell subprocesses to run python scripts

121c325

This was referenced Jun 11, 2026

fix(shell): use PowerShell EncodedCommand for reliable UTF-8 output #31925

Closed

fix(shell): add PowerShell UTF-8 command wrapper on Windows #31985

Closed

Drangeer mentioned this pull request Jun 26, 2026

[Bug] Windows: CJK characters garbled in TUI after partial re-render, fixed by Ctrl+P command palette refresh #34021

Closed

yanyihan-xiaomi mentioned this pull request Jun 28, 2026

fix: force UTF-8 for Windows shell subprocesses to fix CJK mojibake XiaomiMiMo/MiMo-Code#1418

Merged

This was referenced Sep 14, 2026

fix: force UTF-8 output encoding for PowerShell on Windows #49012

Closed

fix: force UTF-8 output encoding for PowerShell on Windows #49124

Closed

xiaozhu1337 mentioned this pull request Oct 3, 2026

fix: PowerShell output encoding for non-ASCII characters on Windows #23636

Closed

thdxr closed this Oct 10, 2026

Navigation Menu

Repository navigation

fix: set default UTF-8 encoding for spawned subprocess on Windows - #31658

fix: set default UTF-8 encoding for spawned subprocess on Windows#31658
Thalynor wants to merge 2 commits into
anomalyco:devfrom
Thalynor:fix/shell-utf8-encoding

Thalynor commented Jun 10, 2026 •

edited

Loading

Uh oh!

Leon0714 commented Aug 12, 2026

Uh oh!

Bigzhangbig commented Sep 29, 2026

Uh oh!

xiaozhu1337 commented Oct 3, 2026

Uh oh!

thdxr commented Oct 10, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Thalynor commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue for this PR

Type of change

What does this PR do?

How did you verify your code works?

Screenshots / recordings

Checklist

Uh oh!

Leon0714 commented Aug 12, 2026

Uh oh!

Bigzhangbig commented Sep 29, 2026

Uh oh!

xiaozhu1337 commented Oct 3, 2026

Uh oh!

thdxr commented Oct 10, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Thalynor commented Jun 10, 2026 •

edited

Loading