fix(cli): write redirected output as UTF-8 so EvoSci does not crash on Windows by Thanatos9404 · Pull Request #663 · EvoScientist/EvoScientist

Thanatos9404 · 2026-10-19T11:58:23Z

Description

Closes #660.

main() now switches stdout and stderr to UTF-8 before anything is printed, as the issue proposes. A PYTHONIOENCODING the user set is kept (an empty one counts as unset, since Python ignores it), and each stream keeps its error handler: reconfigure(encoding=...) alone would reset stderr's backslashreplace to strict.

One change from the proposal: on Windows isatty() is also true for the NUL device, which a child gets from Node's stdio: "ignore", and NUL still gets the code page. So on Windows the switch does not depend on isatty(); a console there is UTF-8 already. Elsewhere a terminal keeps its locale encoding.

As the issue notes, a reader that decodes with the code page (PowerShell's pipe, unless [Console]::OutputEncoding is UTF-8) now gets UTF-8, so box-drawing and other non-ASCII characters can show garbled there. The install scripts keep their PYTHONIOENCODING, since they can install older releases.

Tested: the new tests/test_cli_output_encoding.py fails on main and passes here. On Windows, a script that runs main() with an app printing ✓ exits 1 with the issue's UnicodeEncodeError on main, with stdout sent to a file, a pipe or NUL, and exits 0 with UTF-8 output on this branch.

Type of change

Checklist

I have read the Contributing Guidelines
This targets core functionality used by the majority of users (niche features belong in EvoSkills)
I have added/updated tests where applicable
uv run ruff check . passes
uv run pytest passes (on my Windows machine, 13 tests that need symlink privilege or a git config fail the same way on main)

Summary by CodeRabbit

Bug Fixes
- Improved command-line output compatibility by using UTF-8 for redirected output and on Windows, reducing encoding-related display issues.
- Preserved explicitly configured output encodings and each stream’s error handling.
- Kept terminal streams unchanged where applicable, evaluated stdout and stderr independently, and left streams unchanged when they cannot be reconfigured.

…dows On Windows, Python writes to a pipe, a file or the NUL device in the ANSI code page, so the first character outside it (the check mark after a start, a box-drawing character, a path) stopped the command with UnicodeEncodeError, in EvoSci deploy > deploy.log for example. main() now switches stdout and stderr to UTF-8 before anything is printed, unless the user set PYTHONIOENCODING. Elsewhere a terminal keeps its locale encoding; on Windows every stream is switched, since isatty() is true for NUL as well and a console gets UTF-8 already. Each stream keeps its error handler. Closes EvoScientist#660

… plain pipe

coderabbitai · 2026-10-19T11:58:51Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 7e113205-a083-49b1-b670-df945e09fe0c

📥 Commits

Reviewing files that changed from the base of the PR and between bd84922 and 3257f2f.

📒 Files selected for processing: 1

tests/test_cli_output_encoding.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

📝 Walkthrough

The CLI now configures supported stdout and stderr streams to use UTF-8 before output-producing setup when required by the platform and stream type. A nonempty PYTHONIOENCODING prevents reconfiguration. Tests cover these conditions and handling of streams that cannot be switched.

Changes: CLI output encoding

Layer / File(s)	Summary
Configure and test output streams `EvoScientist/cli/__init__.py`, `tests/test_cli_output_encoding.py`	The CLI configures supported streams to UTF-8 before subsequent setup on Windows or when streams are not terminals, unless `PYTHONIOENCODING` is nonempty. It preserves error handlers and ignores `OSError` and `ValueError`. Tests cover redirected output, platform and TTY conditions, explicit encoding, independent streams, and streams that cannot be reconfigured.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: x-izhang, jfilipiuk

Merge Risk: ⚪ Minimal · up to 3257f

The change makes redirected CLI output use UTF-8 and adds tests for it. No merge-blocking risk is evident from the supplied context.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: configuring redirected CLI output as UTF-8 to prevent Windows crashes.
Linked Issues check		Issue `#660` requires a process-level fix for UnicodeEncodeError in redirected Windows output. `main()` calls `_write_redirected_output_as_utf8()` before CLI output. The helper switches streams to UTF-8…
Out of Scope Changes check		The pull request changes `EvoScientist/cli/__init__.py` and adds `tests/test_cli_output_encoding.py`. The implementation and tests directly support issue `#660`. No unrelated change appears in the revie…

Full details: Linked Issues check

Explanation

Issue #660 requires a process-level fix for UnicodeEncodeError in redirected Windows output. main() calls _write_redirected_output_as_utf8() before CLI output. The helper switches streams to UTF-8 when PYTHONIOENCODING is unset, treats an empty value as unset, preserves each stream's error handler, and handles Windows NUL behavior. tests/test_cli_output_encoding.py covers redirected streams, terminals, explicit encoding, independent stream decisions, and unswitchable streams.

Full details: Out of Scope Changes check

Explanation

The pull request changes EvoScientist/cli/__init__.py and adds tests/test_cli_output_encoding.py. The implementation and tests directly support issue #660. No unrelated change appears in the reviewed diff.

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

CI annotation context is incomplete: annotation collection unavailable or limited

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

jfilipiuk

Nice work, thank you!

X-iZhang

Thanks for the fix and for checking it on Windows with a file, a pipe and NUL. One suggestion on the tests, inline in tests/test_cli_output_encoding.py.

Thanatos9404 added 2 commits October 19, 2026 17:09

fix(cli): say in the docstring which streams are switched, and test a…

bd84922

… plain pipe

din0s added bug Something isn't working external contribution Implementation from an external contributor labels Oct 19, 2026

X-iZhang requested a review from jfilipiuk October 19, 2026 13:21

jfilipiuk approved these changes Oct 19, 2026

View reviewed changes

X-iZhang reviewed Oct 19, 2026

View reviewed changes

Comment thread tests/test_cli_output_encoding.py Resolved

Update tests/test_cli_output_encoding.py

3257f2f

X-iZhang approved these changes Oct 19, 2026

View reviewed changes

X-iZhang merged commit 16944b9 into EvoScientist:main Oct 19, 2026
17 checks passed

X-iZhang mentioned this pull request Oct 24, 2026

release: v0.3.8 #685

Merged

9 of 10 tasks

Navigation Menu

Repository navigation

fix(cli): write redirected output as UTF-8 so EvoSci does not crash on Windows - #663

fix(cli): write redirected output as UTF-8 so EvoSci does not crash on Windows#663
X-iZhang merged 3 commits into
EvoScientist:mainfrom
Thanatos9404:fix/660-utf8-redirected-output

Thanatos9404 commented Oct 19, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

jfilipiuk left a comment

Uh oh!

X-iZhang left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Thanatos9404 commented Oct 19, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jfilipiuk left a comment

Choose a reason for hiding this comment

Uh oh!

X-iZhang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Thanatos9404 commented Oct 19, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading