Skip to content

Pass through line rendition attributes over conpty#13933

Merged
2 commits merged intomicrosoft:mainfrom
j4james:conpty-line-rendition
Sep 9, 2022
Merged

Pass through line rendition attributes over conpty#13933
2 commits merged intomicrosoft:mainfrom
j4james:conpty-line-rendition

Conversation

@j4james
Copy link
Collaborator

@j4james j4james commented Sep 6, 2022

This PR introduces a mechanism for passing through line rendition
attributes to the conpty client, so we can support double-width and
double-height text in Windows Terminal.

Line renditions were first implemented in conhost (with the GDI
renderer) in PR #8664, and were implemented in the DX renderer in PR
#13102.

By default this won't add any additional overhead to the conpty output,
but as soon as there is any usage of double-size text, we switch to a
mode in which every line output will be prefixed with a line rendition
sequence. This is to ensure that the line attributes in the client
terminal are always in sync with the host.

Since this does add some overhead to the conpty output, we'd prefer not
to remain in this mode longer than necessary. So whenever there is a
full repaint of the entire viewport, we check to see if all of the lines
are single-width. If that is the case, we can then safely skip the line
rendition sequences in future updates.

One other small optimization is when the conpty update is only writing
out a single character (this is something we already check for). When
that is the case, we can safely skip the line rendition prefix, because
a single character update should never include a change of the line
rendition.

Validation Steps Performed

I've manually tested that Windows Terminal now passes the double-size
tests in Vttest, and also confirmed various edge cases are working
correctly in my own double-size tests.

Closes #11595

@ghost ghost added Area-Rendering Text rendering, emoji, complex glyph & font-fallback issues Area-VT Virtual Terminal sequence support Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Product-Terminal The new Windows Terminal. labels Sep 6, 2022
@DHowett
Copy link
Member

DHowett commented Sep 6, 2022

It's unfortunate that we're keen on replacing the DX engine here! Thanks for doing all of this. @lhecker we should consider how we'll represent glyphs in the atlas that take up more vertical space than one row (!)

@j4james
Copy link
Collaborator Author

j4james commented Sep 6, 2022

It's unfortunate that we're keen on replacing the DX engine here! Thanks for doing all of this. @lhecker we should consider how we'll represent glyphs in the atlas that take up more vertical space than one row (!)

I haven't really looked at the Atlas renderer, because I assumed I would be a bit out of my depth there, but I had hoped that the line-rendition support I added to the DX renderer could be ported fairly easily by someone that knew what they were doing.

And I wouldn't think you'd need to do anything special in the atlas if you're taking the same approach - it's just a transform that's applied when each line is rendered.

@j4james j4james marked this pull request as ready for review September 6, 2022 19:10
@DHowett
Copy link
Member

DHowett commented Sep 6, 2022

And I wouldn't think you'd need to do anything special in the atlas if you're taking the same approach - it's just a transform that's applied when each line is rendered.

Possibly. The core conceit of atlas is that we rasterize each [glyph * font] combo one time into a GPU texture and then use a pixel shader to copy and color the glyph mask directly to the render target. I think if we approach line renditions the same way as we do in DX, we may end up with scaling artifacts.

My gut tells me that we'll want to do the transform in the early phase when we're producing the glyph texture, and somehow mark that glyph as requiring multiple cells much like we would if it were multi-width. Hmm...

@j4james
Copy link
Collaborator Author

j4james commented Sep 6, 2022

I think if we approach line renditions the same way as we do in DX, we may end up with scaling artifacts.

Yeah, I just realised that after I posted my reply. But I'd personally be perfectly happy with that as a solution. It's certainly better than nothing. Obviously this is just me, but right now most of my use cases for double-size text involve soft fonts, which have a fixed resolution anyway, so they're always going to be scaled.

On that note, I should mention that I have a plan for soft font support over conpty which I'd like to create a PR for at some point. And again I'm hoping someone smarter than me will be able to port the DX implementation to Atlas so we can have soft fonts working there too.

@j4james
Copy link
Collaborator Author

j4james commented Sep 6, 2022

My gut tells me that we'll want to do the transform in the early phase when we're producing the glyph texture, and somehow mark that glyph as requiring multiple cells much like we would if it were multi-width.

Thinking about this some more, double-height glyphs may not actually be a problem, because you only ever render one half of the glyph at a time (i.e. the top is rendered independently of the bottom). So the only additional functionality I think you might need would be some extension to the ID to distinguish between variants of a particular character (single width, double-width, double-height top, double-height bottom).

And this is assuming the atlas can already handle glyphs that are multiple cell widths, and not just double width, because any existing double width character, would now potentially be quadruple width.

@DHowett
Copy link
Member

DHowett commented Sep 6, 2022

Ah, right! It's actually just the tops and bottoms of the glyphs cleverly rendered next to eachother.

We do have support for glyphs that are >2 cells in width today; that's how we ended up implementing ligatures! That does mean that Cascadia's "infinite arrow" ligatures are, ah, expensive in terms of glyph space... but what can you do?

Glyph atlas with an infinite arrow

Each version of the ligature is rendered in full into the texture.

screen

screen

atlas

image

@lhecker
Copy link
Member

lhecker commented Sep 6, 2022

Yep I'll gladly port everything over to AtlasEngine if you don't feel comfortable with it. Now that the new engine is sorta useable, I'm currently in the process of cleaning up that project to be better structured and easier to deal with.

For context:

  • cells: A 3 cell wide glyph reserves 3 equally large cells in the texture atlas. The cells array stores those texture coordinates. The cells in the texture atlas aren't necessary next to each other, but the pixel shader will bit-blit them to appear next to each other on the screen.
  • cellCount: The number of cells a glyph owns (in this example it's 3).
  • CellFlags: Stores glyph and cell-specific attributes, like is-italic/intense/underlined/overlined/etc.

I'm planning to add double-height glyphs as a flag to CellFlags. If that flag is set, then the cells array stores twice as many entries as the cellCount indicates. All the existing code that assumes that the cellCount equals the width of the glyph in cells will continue to work. But if we encounter a double-height-line-rendition row then we will use the second half of the cells array to copy over another cellCount-many cells over into the lower half of the double-height row.

@j4james
Copy link
Collaborator Author

j4james commented Sep 6, 2022

But if we encounter a double-height-line-rendition row then we will use the second half of the cells array to copy over another cellCount-many cells over into the lower half of the double-height row.

Just FYI, that's not how double-height rendition works. The top and bottom are completely independent and don't necessarily match. So you can have the top half of the letters ABC over the bottom half of XYZ. Or say one line being just the bottom half of the characters with single height lines above and below. You can mix and match line attributes however you want.

So in the same way that you have variants of each character for italic and bold, etc. I'd assume you'd also have double-width, double-height top, and double-height bottom variants. Then when rendering a line with the DoubleHeightTop attribute, you'd use the corresponding top variant for all the characters in that line. What appears on the following line is irrelevant.

Copy link
Member

@DHowett DHowett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Now that 1.16 has branched, we're clear to land new fun stuff.

@DHowett DHowett added the AutoMerge Marked for automatic merge by the bot when requirements are met label Sep 9, 2022
@ghost
Copy link

ghost commented Sep 9, 2022

Hello @DHowett!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit 3e6abd3 into microsoft:main Sep 9, 2022
@j4james j4james deleted the conpty-line-rendition branch September 13, 2022 21:10
@ghost
Copy link

ghost commented Jan 24, 2023

🎉Windows Terminal Preview v1.17.1023 has been released which incorporates this pull request.:tada:

Handy links:

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Rendering Text rendering, emoji, complex glyph & font-fallback issues Area-VT Virtual Terminal sequence support AutoMerge Marked for automatic merge by the bot when requirements are met Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Product-Terminal The new Windows Terminal.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Double-Height and Double-Width text to Windows Terminal

3 participants