Skip to content

bug: CJK char corruption in text rendering #255

New issue
New issue
Closed
#307
Closed
bug: CJK char corruption in text rendering#255
#307
Labels
bugSomething isn't workingcoreThis relates to the core package

Description

@zenyr
zenyr
opened on Nov 3, 2025

Problem

Both Solid and React reconcilers corrupt certain CJK characters during rendering. Core handles them correctly.

Expected → Actual (Solid/React)
你好世界 → 你好世ç界
中文   → 中文æ
한글   → í한글

Emoji & ASCII unaffected.

Root Cause

Encoding/displayWidth mismatch in JSX→TextNode conversion path. Likely culprit: reconciler text handling vs core displayWidth calc.

Investigation Notes

  • Core library handles all CJK/emoji correctly
  • Issue affects both Solid and React reconciler text rendering paths
  • Reproduction: specific CJK chars render as different glyphs

Next Steps

  1. Debug TextNode creation from Solid/React JSX
  2. Verify displayWidth consistency with core
  3. Check for encoding issues in reconciler text handling

Created with assistance from OpenCode & Claude Haiku 4.5

Activity

  1. zenyr
    changed the title [-]fix(solid): CJK char corruption in text rendering[/-] [+]fix: CJK char corruption in text rendering[/+] on Nov 3, 2025

  2. kommander commented on Nov 3, 2025

    @kommander
    kommander
    on Nov 3, 2025
    Collaborator

    Thanks for looking into this. I think the root cause for something like this would rather be in the renderer.zig for the ansi output, or width calculations in utf8.zig. Setting a default background prevents text from ever being rendered on transparent terminal background.

  3. zenyr
    added 3 commits that reference this issue on Nov 3, 2025

    fix: skip rendering continuation cells to fix CJK character corruption

    5e0ab97

    fix: ensure cursor position sync after rendering wide characters

    0c46e61

    fix: CJK character corruption by fixing wide character rendering and …

    bf0df7f

  4. kommander commented on Nov 3, 2025

    @kommander
    kommander
    on Nov 3, 2025
    Collaborator
    Image

    I can't seem to be able to reproduce that. What terminal are you using?

  5. zenyr commented on Nov 4, 2025

    @zenyr
    zenyr
    on Nov 4, 2025
    Author

    Oh, okay. I'll attach my system configuration here:
    I also found this issue on Windows 11 Powershell / CMD environment too.
    I've made a tangent PR (adding React test suite with skipped tests) on #262.

    ❯ fastfetch
                     ..'          jinhyeok
                 ,xNMM.           -----------------------------------------
               .OMMMMo            OS: macOS Sequoia 15.6.1 arm64
               lMM"               Host: MacBook Pro (14-inch, 2023)
     .;loddo:.  .olloddol;.       Kernel: Darwin 24.6.0
   cKMMMMMMMMMMNWMMMMMMMMMM0:     Uptime: 7 days, 19 hours, 46 mins
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.     Packages: 230 (brew), 16 (brew-cask)
 XMMMMMMMMMMMMMMMMMMMMMMMX.       Shell: zsh 5.9
;MMMMMMMMMMMMMMMMMMMMMMMM:        Display (Color LCD): 3024x1964 @ 120 Hz (as 1512x982) in 14" [Built-in] *
:MMMMMMMMMMMMMMMMMMMMMMMM:        Display (RTK UHD HDR): 3360x1890 @ 60 Hz (as 1680x945) in 24" [External]
.MMMMMMMMMMMMMMMMMMMMMMMMX.       Display (LG HDR 4K): 3840x2160 @ 60 Hz (as 1920x1080) in 27" [External]
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.     Display (LG HDR 4K): 3840x2160 @ 60 Hz (as 1920x1080) in 27" [External]
 'XMMMMMMMMMMMMMMMMMMMMMMMMMMk    DE: Aqua
  'XMMMMMMMMMMMMMMMMMMMMMMMMK.    WM: Quartz Compositor 278.4.7
    kMMMMMMMMMMMMMMMMMMMMMMd      WM Theme: Multicolor (Light)
     ;KMMMMMMMWXXWMMMMMMMk.       Font: .AppleSystemUIFont [System], Helvetica [User]
       "cooc*"    "*coo'"         Cursor: Fill - Black, Outline - White (32px)
                                  Terminal: tmux 3.4
                                  CPU: Apple M2 Max (12) @ 3.50 GHz
                                  GPU: Apple M2 Max (38) @ 1.40 GHz [Integrated]
❯ ghostty --version
Ghostty 1.3.0-main+3f75c66e8

Version
  - version: 1.3.0-main+3f75c66e8
  - channel: tip
Build Config
  - Zig version   : 0.15.2
  - build mode    : .ReleaseFast
  - app runtime   : .none
  - font engine   : .coretext
  - renderer      : renderer.generic.Renderer(renderer.Metal)
  - libxev        : kqueue

    Update:

    • I'm not sure if this is related but this happened twice in a row for me, which is weird. (that t)
    Image
  6. zenyr
    changed the title [-]fix: CJK char corruption in text rendering[/-] [+]bug: CJK char corruption in text rendering[/+] on Nov 4, 2025

  7. kommander commented on Nov 4, 2025

    @kommander
    kommander
    on Nov 4, 2025
    Collaborator

    Ahh tmux might be the culprit, it has some Unicode quirks, will try to reproduce.

    The issue with the overlapping "t" is a different one, that needs flexShrink=0 and is handled in an issue on opencode.

  8. zenyr commented on Nov 5, 2025

    @zenyr
    zenyr
    on Nov 5, 2025
    Author

    tmux might be the culprit

    Hmm while I do agree with the reasoning I could 100% reproduce this on my Windows 11 device, which does not have any tmux related stuff available.

  9. kommander commented on Nov 5, 2025

    @kommander
    kommander
    on Nov 5, 2025
    Collaborator

    Yes, the windows terminals calculate Unicode width differently and not complete to the standard as well, like tmux. Proper Unicode support is lacking in many implementations unfortunately.

  10. kommander commented on Nov 6, 2025

    @kommander
    kommander
    on Nov 6, 2025
    Collaborator
    Image I saw some of it happening. Happens at wrapping break points, it might calculate the byte offset for the chars wrong there. Similarly, for streaming content when the text is incomplete it might have bytes until the middle of a grapheme.

  11. zenyr commented on Nov 6, 2025

    @zenyr
    zenyr
    on Nov 6, 2025
    Author

    Ah sure, Windows native terminal is notorious for CJK users already :)

    Hoever,I think I could reliably reproduce grapheme error, with or without chunk streaming.

    For example:

    Image

    알겠습니다. Task 에이전트에 ktlint + detekt 검사를 위임하겠습니다.

    This rendering issue persists regardless of being streamed or not.
    This is another screenshot of the same session loaded without tmux: (opencode --continue)

    Image
  12. kommander
    added
    bugSomething isn't working
    coreThis relates to the core package
    on Nov 7, 2025

  13. 12 remaining items

  14. kommander commented on Nov 18, 2025

    @kommander
    kommander
    on Nov 18, 2025
    Collaborator

    Alright, let's do this again, I'll get it right at some point. So many edge cases.

  15. kommander commented on Dec 10, 2025

    @kommander
    kommander
    on Dec 10, 2025
    Collaborator

    I haven't forgotten about this, just didn't get through to it yet.

  16. msmps
    added 2 commits that reference this issue on Jan 8, 2026

    fix multibyte char split in wrapping logic (#307)

    61c7c89

    fix word wrapping with CJK and at wrap/chunk boundaries (#319)

    b4c3d52

  17. kskang commented on Jan 11, 2026

    @kskang
    kskang
    on Jan 11, 2026

    I created a PR that may fix this issue: #512

    The root cause seems to be char_offset vs col_offset mismatch in word wrap boundary detection. CJK characters have display width 2, but the code was using character count (width 1) for wrap calculations.

    Would appreciate if someone could test this fix.

  18. kommander commented on Jan 18, 2026

    @kommander
    kommander
    on Jan 18, 2026
    Collaborator

    This should be better now?

  19. zenyr commented on Jan 20, 2026

    @zenyr
    zenyr
    on Jan 20, 2026
    Author

    https://opncd.ai/share/73TBWAXf (Last message contains a summary)

    It's MUCH MUCH better now, the only edge case that I found was the Keycap emojis, which Haiku 4.5 often uses.
    Other than that I think... it's fixed!

  20. zenyr
    closed this as completedon Jan 20, 2026

  21. zenyr commented on Jan 20, 2026

    @zenyr
    zenyr
    on Jan 20, 2026
    Author

    Thanks for fixing up the majority of issues. Please refer to the shared session above for the remaining edge cases. @kommander

  22. zenyr
    added 6 commits that reference this issue on Jan 30, 2026

    fix(core): use byte offset for line_starts in text wrap

    aedc3b7

    fix(core): use byte offset for line_starts in text wrap

    10ee149

    fix(core): use byte offset for line_starts in text wrap

    841364c

    test(core): add no-wrap byte offset test for CJK text

    4f2e8d1

    fix(core): use byte offset for line_starts in no-wrap mode

    00336eb

    perf(core): store byte_start/byte_len in VirtualChunk

    e344f44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcoreThis relates to the core package

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions