Skip to content

fix(cli): force UTF-8 stdout/stderr on Windows console (#643 cluster H-1)#728

Merged
memtomem merged 1 commit intomainfrom
fix/cli-utf8-windows-stdout
May 2, 2026
Merged

fix(cli): force UTF-8 stdout/stderr on Windows console (#643 cluster H-1)#728
memtomem merged 1 commit intomainfrom
fix/cli-utf8-windows-stdout

Conversation

@memtomem
Copy link
Copy Markdown
Owner

@memtomem memtomem commented May 2, 2026

Summary

Cluster H-1 of #643 — Windows console default codepage (cp1252/cp437) cannot encode the box-drawing and em-dash glyphs the wizard prints, so mm init crashes at the very first banner line on a fresh Windows runner.

This is a production bug, not a test-only artifact: every Windows user hits it on the first mm init invocation.

Stack from the failing CI run (sha 88c3e81):

init_cmd.py:2781  click.secho("  ─────────────", fg="cyan")
click/utils.py:321  file.write(out)
UnicodeEncodeError: 'charmap' codec can't encode character '─' (U+2500)

Approach

Spot-fixing line 2781 leaves every other unicode glyph (em-dash, future additions) as a latent crash. Reconfigure sys.stdout / sys.stderr at the CLI entry point (the cli() group callback runs before any subcommand emits output), gated on sys.platform == "win32" so POSIX is a no-op:

if sys.platform == "win32":
    for stream in (sys.stdout, sys.stderr):
        try:
            stream.reconfigure(encoding="utf-8", errors="replace")
        except (AttributeError, io.UnsupportedOperation):
            pass

errors="replace" degrades a missing glyph to ? instead of crashing — strictly better than the current behavior.

Test plan

  • POSIX unchanged: uv run pytest -m "not ollama" packages/memtomem/tests/test_cli.py packages/memtomem/tests/test_cli_status.py packages/memtomem/tests/test_init_cmd.py — 355 passed.
  • uv run pytest -m "not ollama" packages/memtomem/tests/test_cli_index_noop_e2e.py — 2 passed.
  • uv run ruff check + uv run ruff format --check on the touched file.
  • CI test (windows-latest) will turn test_init_index_search_via_subprocess from FAIL → PASS (cluster H-1 of the Windows compat sweep).

Cluster H-2 (test_force_overrides_db_lock) is a separate root cause and ships in a follow-up PR.

🤖 Generated with Claude Code

…H-1)

The Windows console default codepage (cp1252/cp437) cannot encode the
box-drawing and em-dash glyphs the wizard prints. On a fresh Windows
runner ``mm init`` crashes at the very first banner line:

    init_cmd.py:2781  click.secho("  ─────────────")
    click/utils.py:321  file.write(out)
    UnicodeEncodeError: 'charmap' codec can't encode character '─'

This is a real production bug — every Windows user hits it on the first
``mm init`` invocation, not just CI. Spot-fixing line 2781 leaves every
other unicode glyph in the wizard (em-dash, future additions) as a
latent crash, so reconfigure stdout/stderr at the entry point instead.

The reconfigure happens in the click group callback so it runs before
any subcommand emits output, and is gated on ``sys.platform == "win32"``
so POSIX is a no-op. ``errors="replace"`` degrades a missing glyph to
``?`` instead of crashing — strictly better than the current behavior.

Verification:
- ``test (windows-latest)`` will now pass
  ``test_init_index_search_via_subprocess`` (cluster H-1).
- POSIX unchanged: ``uv run pytest -m "not ollama"
  packages/memtomem/tests/test_cli*.py packages/memtomem/tests/test_init_cmd.py``
  (362 passed) and ``test_cli_index_noop_e2e.py`` (2 passed).

Co-Authored-By: Claude <[email protected]>
@memtomem
Copy link
Copy Markdown
Owner Author

memtomem commented May 2, 2026

CI status — Windows fail is carried-over, not from this PR

test (windows-latest) is red, but the failure this PR was meant to fix is gone: test_init_index_search_via_subprocess is no longer in the failure list. The 9 fails on the run are all unrelated:

Net effect of this PR alone: cluster H-1 (test_init_index_search_via_subprocess) goes from FAIL → PASS, the other 9 fails are inherited from main. Once #729 lands, only cluster A remains.

Admin merge (feedback_green_ci_admin_merge.md) is appropriate here — required check is red but the redness is unrelated to this diff.

@memtomem memtomem merged commit c7fcfa8 into main May 2, 2026
8 of 9 checks passed
@memtomem memtomem deleted the fix/cli-utf8-windows-stdout branch May 2, 2026 11:26
@github-actions github-actions Bot locked and limited conversation to collaborators May 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants