Skip to content

Add configurable ambiguous-width policy (narrow/wide)#19864

Merged
DHowett merged 6 commits intomicrosoft:mainfrom
unagi:dev/ray/ambiguous-width-narrow-wide
Feb 27, 2026
Merged

Add configurable ambiguous-width policy (narrow/wide)#19864
DHowett merged 6 commits intomicrosoft:mainfrom
unagi:dev/ray/ambiguous-width-narrow-wide

Conversation

@unagi
Copy link
Contributor

@unagi unagi commented Feb 13, 2026

This PR introduces compatibility.ambiguousWidth as a global
compatibility setting (narrow default, wide optional).

The default remains narrow.

Why global-only in this PR:

  • Width detection is currently process-wide (CodepointWidthDetector
    singleton).
  • True profile-level ambiguous-width behavior would require broader
    architectural changes and is intentionally deferred to a follow-up
    design/PR.

What this PR guarantees:

  • Terminal-side handling is consistent end-to-end for the selected
    ambiguous-width policy (rendering path + ConPTY/host propagation).

Known limitation:

  • Some client applications (for example PSReadLine/readline-based apps)
    may still compute character widths independently.
  • In such cases, cursor movement or Backspace behavior can differ from
    visual cell width even when terminal-side policy is consistent.

This is a compatibility/readability trade-off feature:

  • narrow: prioritize cross-application compatibility.
  • wide: prioritize readability with many CJK fonts.

Closes #153
Closes #370
Refs #2928
Refs #2049, #2066, #2375, #900, #5910, #5914

Co-authored-by: Leonard Hecker [email protected]

Reason:\n- Address ambiguous-width rendering requests discussed in microsoft#153 and microsoft#370 without introducing a locale-based default policy.\n\nScope:\n- Add compatibility.ambiguousWidth as narrow|wide and wire it through settings model/schema/UI.\n- Propagate the selected mode through Terminal -> ConPTY -> winconpty -> conhost argument paths.\n- Replace fixed ambiguous width handling in CodepointWidthDetector with policy-based resolution and keep related width paths consistent.\n- Extend regression coverage for ambiguous symbols, DEC special-character alignment, fallback lifecycle stability, and emoji/VS16-adjacent behavior.\n\nBreaking changes:\n- None. Default behavior remains narrow.
@unagi
Copy link
Contributor Author

unagi commented Feb 13, 2026

@microsoft-github-policy-service agree

@lhecker
Copy link
Member

lhecker commented Feb 13, 2026

Can you please post a screenshot of your build showing how it works with the ambiguous width character "…"?

If you're on the latest Windows version, testing it is very easy: Simply run edit.exe and open the "File" context menu. It should look like this, but with a wide "…" character:

image

@lhecker lhecker added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Feb 13, 2026
@unagi
Copy link
Contributor Author

unagi commented Feb 14, 2026

@lhecker

As requested, here are screenshots showing the behavior before and after my change.

Settings UI

This shows the new ambiguous width option in the compatibility settings.

image

After fix (Ambiguous width = Wide)

The arrow character (, EastAsianWidth=Ambiguous) is measured as two cells.
The cursor position, spacing, and following characters are aligned correctly.

image

Before / Narrow mode

With the current default behavior (Ambiguous treated as narrow),
the same character is measured as one cell while the glyph visually occupies
almost two cells in typical CJK fonts.
This causes the next character to overlap and the cursor position to drift.

image

This demonstrates that allowing an explicit ambiguous-width policy
improves layout correctness in CJK environments without changing the default behavior.

@microsoft-github-policy-service microsoft-github-policy-service bot added Needs-Attention The core contributors need to come back around and look at this ASAP. and removed Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Feb 14, 2026
Reason: review pointed out unnecessary newline code changes in PR microsoft#19864.

Impact: keep logical changes intact while normalizing edited files to repository line endings.

Breaking change: none.
@german-one
Copy link
Contributor

Fonts may have an influence. Perhaps this is the reason why you are facing overlapping glyphs, which I was not even able to reproduce (FWIW, Cascadia Mono in my case).

Besides of that, my observation is that PowerShell 5.1 in particular bahaves weirdly. As your screenshots seem to demonstrate how it looks like in PS 5.1 only, I'm wondering what this setting does to the experience in other shells?

I tried to have the cursor visible in the screen captures, just to show that the "A" is written adjacent to the last arrow, no matter if the cursor has been already advanced to an even farther position in PS 5.1.

PS 7.5
arrow_terminal_ps75

PS 5.1
arrow_terminal_ps51

Other shells (CMD, WSL) do not misplace the cursor. The behavior of PS 5.1 can also be reproduced in a conhost window though.

To be clear, I'm not questioning this new setting. And I also don't know how much my observations interact with specifics of CJK scripts in general. However, it might be more reasonable to make it a profile setting

  • if the appearance differs depending on the shell used
  • if it may cause misalignments in text mode interfaces like in the menu of @lhecker's screenshot or in apps like Far Manager

What do you think?

… policy

Reason: when --textMeasurement is omitted, conpty init could leave width recognition mode implicit, causing ambiguity-width related drift.

Scope: always initialize VtIo text measurement to graphemes by default and add regression tests for default/explicit textMeasurement+ambiguousWidth propagation.

Breaking changes: none.
@unagi unagi force-pushed the dev/ray/ambiguous-width-narrow-wide branch from 82d9bad to 4a09610 Compare February 17, 2026 06:08
@unagi
Copy link
Contributor Author

unagi commented Feb 17, 2026

@german-one
Thank you for the feedback. Let me add the context and intent.

In real CJK usage, some users choose fonts that render East Asian Ambiguous characters as width 2 for readability. This PR provides wide as a relief option for that kind of environment. At the same time, the default remains narrow, so compatibility-first behavior is preserved by default (wide is explicit opt-in).

We also recognize the side effect that cursor/Backspace behavior can diverge in readline-based environments. That behavior depends on how the receiving application computes character width. We are not hiding this; it is now explicitly documented as a known limitation in both the HelpText and the PR description.

So the intent of this PR is to provide an explicit option for readability-focused cases, while clearly disclosing the tradeoff.

I also attached reference screenshots (HelpText reinforcement in Settings, plus the known readline-side behavior).

image image

@BlueRain-debug
Copy link

@german-one Thank you for the feedback. Let me add the context and intent.

In real CJK usage, some users choose fonts that render East Asian Ambiguous characters as width 2 for readability. This PR provides wide as a relief option for that kind of environment. At the same time, the default remains narrow, so compatibility-first behavior is preserved by default (wide is explicit opt-in).

We also recognize the side effect that cursor/Backspace behavior can diverge in readline-based environments. That behavior depends on how the receiving application computes character width. We are not hiding this; it is now explicitly documented as a known limitation in both the HelpText and the PR description.

So the intent of this PR is to provide an explicit option for readability-focused cases, while clearly disclosing the tradeoff.

I also attached reference screenshots (HelpText reinforcement in Settings, plus the known readline-side behavior).

image image

I think it would be better to add a third option (auto?) to automatically determine the width based on the current font (similar to Conhost's behavior)? ofc I admit that implementing this might involve some work.

@DHowett
Copy link
Member

DHowett commented Feb 20, 2026

I think it would be better to add a third option (auto?)

We are absolutely not going to do that. That was one of the biggest mistakes the original Windows Console made, and it means that standards-conforming applications cannot accurately predict how big something is going to be without knowing--and manually parsing--the font. Accessing the font is impossible over a remote connection (for example, and among other reasons.)

If you want that behavior, you can stick with the original Windows Console. 🙂

@unagi
Copy link
Contributor Author

unagi commented Feb 21, 2026

Thanks everyone for the feedback and discussion.

I agree that an automatic/font-dependent mode is probably not the right direction, since it can make character width less predictable for applications (especially over remote connections).

What this PR is trying to do is more limited: keep today’s default behavior as-is, while offering an explicit opt-in for users/environments (including some CJK setups) that prefer ambiguous characters to be treated as wide.

So the intent here is explicit selection with clear tradeoffs (narrow / wide), rather than adding an auto mode.

@j4james
Copy link
Collaborator

j4james commented Feb 21, 2026

FYI, there is a mode supported by some terminals to switch between narrow and wide for ambiguous width characters. That at least allows applications to make the choice to opt in if they're prepared to handle the non-standard widths.

That said, you may still want the option to allow users to force the mode on, and I'm in agreement with @german-one that it should probably be a per-profile setting. I understand that there may be architectural reasons why this isn't feasible.

ETA: The mode number I'm referring to is ?8428. When set, ambiguous width is narrow; when reset, it's wide. The only terminals I'm aware of that support this are MLTerm and RLogin.

Copy link
Member

@lhecker lhecker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unagi I have finished reviewing your PR. First off, thank you for submitting this - you did a great job creating this! The code was in all the right places and I did not have to add anything. (I did remove some things though.)

I have pushed a commit that simplifies your logic a bit. For instance, instead of passing --ambiguousWidth narrow/wide it just passes a boolean flag --ambiguousIsWide. I also rewrote the settings description slightly to better fit the existing tone (personal opinion). All of that are just minor issues though.

But I also have one actual critique: Your PR was written by AI, and that's absolutely fine. However, we expect contributors to read & understand their submission too. There were some spots where I'm quite certain that you didn't review it yourself yet. For instance, the unit tests asserted on the width of U+2666 ♦ being ambiguous width. That character is documented as "neutral" width which means it's always narrow. Another example is the rewrite of the CodepointWidthDetector which broke the TextMeasurementMode::Console mode. The good news is that I fixed these instances and the PR is a lot smaller now.

@lhecker
Copy link
Member

lhecker commented Feb 26, 2026

/azp run

@lhecker lhecker requested a review from DHowett February 26, 2026 22:13
@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Member

@DHowett DHowett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @lhecker

<comment>A label for a drop down selector, referring to the East Asian Ambiguous Width Unicode specification.</comment>
</data>
<data name="Globals_AmbiguousWidth.HelpText" xml:space="preserve">
<value>Sets the width used for East Asian Ambiguous characters. "Narrow" (default) prioritizes compatibility, while "Wide" prioritizes readability with many CJK fonts. In "Wide" mode, some applications may not work correctly and cause erratic cursor movement. Changing this setting requires a restart of Windows Terminal and it only applies to applications launched from within it.</value>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "East Asian" the correct term of art here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's appropriate, because the spec is called such.

Co-authored-by: Dustin L. Howett <[email protected]>
@DHowett
Copy link
Member

DHowett commented Feb 27, 2026

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@DHowett DHowett enabled auto-merge (squash) February 27, 2026 19:29
@microsoft-github-policy-service microsoft-github-policy-service bot added the Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. label Feb 27, 2026
@microsoft-github-policy-service microsoft-github-policy-service bot added Area-Rendering Text rendering, emoji, complex glyph & font-fallback issues Area-Server Down in the muck of API call servicing, interprocess communication, eventing, etc. Area-Settings Issues related to settings and customizability, for console or terminal Product-Conhost For issues in the Console codebase Product-Conpty For console issues specifically related to conpty Product-Terminal The new Windows Terminal. labels Feb 27, 2026
@DHowett DHowett merged commit 3b8c560 into microsoft:main Feb 27, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Rendering Text rendering, emoji, complex glyph & font-fallback issues Area-Server Down in the muck of API call servicing, interprocess communication, eventing, etc. Area-Settings Issues related to settings and customizability, for console or terminal Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Attention The core contributors need to come back around and look at this ASAP. Product-Conhost For issues in the Console codebase Product-Conpty For console issues specifically related to conpty Product-Terminal The new Windows Terminal.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ambiguous width character in CJK environment Add an option to toggle the width of ambiguous-width characters

6 participants