Summary of the Pull Request

This PR moves the color table and related render settings, which are common to both conhost and Windows Terminal, into a shared class that can be accessed directly from the renderer. This avoids the overhead of having to look up these properties via the IRenderData interface, which relies on inefficient virtual function calls.

This also introduces the concept of color aliases, which determine the position in the color table that colors like the default foreground and background are stored. This allows the option of mapping them to one of the standard 16 colors, or to have their own separate table entries.

References

This is a continuation of the color table refactoring started in #11602 and #11784. The color alias functionality is a prerequisite for supporting a default bold color as proposed in #11939. The color aliases could also be a way for us to replace the PowerShell color quirk for #6807.

PR Checklist

• Closes Introduce a shared class for managing render settings #12002
• CLA signed.
• Tests added/passed
• Documentation updated.
• Schema updated.
• I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Detailed Description of the Pull Request / Additional comments

In addition to the color table, this new RenderSettings class manages the blinking state, the code for adjusting indistinguishable colors, and various boolean properties used in the color calculations. These boolean properties are now stored in a til::enumset so they can all be managed through a single SetRenderMode API, and easily extended with additional modes that we're likely to need in the future.

In Windows Terminal we have an instance of RenderSettings stored in the Terminal class, and in conhost it's stored in the Settings class. In both cases, a reference to this class is passed to the Renderer constructor, so it now has direct access to that data. The renderer can then pass that reference to the render engines where it's needed in the UpdateDrawingBrushes method.

This means the renderer no longer needs the IRenderData interface to access the GetAttributeColors, GetCursorColor, or IsScreenReversed methods, so those have now been removed. We still need access to GetAttributeColors in certain accessibility code, though, so I've kept that method in the IUIAData interface, but the implementation just forwards to the RenderSettings class.

The implementation of the RenderSettings::GetAttributeColors method is loosely based on the original Terminal code, only the CalculateRgbColors call has now been incorporated directly into the code. This let us deduplicate some bits that were previously repeated in the section for adjusting indistinguishable colors. The last steps, where we calculate the alpha components, have now been split in to a separate GetAttributeColorsWithAlpha method, since that's typically not needed.

Validation Steps Performed

There were quite a lot changes needed in the unit tests, but they're mostly straightforward replacements of one method call with another.

In the TextAttributeTests, where we were previously testing the CalculateRgbColors method, we're now running those tests though RenderSettings::GetAttributeColors, which incorporates the same functionality. The only complication is when testing the IntenseIsBright option, that needs to be set with an additional SetRenderMode call where previously it was just a parameter on CalculateRgbColors.

In the ScreenBufferTests and TextBufferTests, calls to LookupAttributeColors have again been replaced by the RenderSettings::GetAttributeColors method, which serves the same purpose, and calls to IsScreenReversed have been replaced with an appropriate GetRenderMode call. In the VtRendererTests, all the calls to UpdateDrawingBrushes now just need to be passed a reference to a RenderSettings instance.