WriteConsoleA in CP_UTF8 cannot handle receiving lead, continuation and trail in separate operations #4791
Description
In til::u8state, when we decrement backIter, we get a checked iterator exception (in debug mode). In release mode, we just throw away the partial because it fails the "if lead byte found" check.
Throw on --backiter:
terminal/src/inc/til/u8u16convert.h
Line 88 in 2d707f1
Here is a test that demonstrates the failing behavior:
void Utf8Utf16ConvertTests::TestU8ToU16OneByOne()
{
const std::string u8String1_1{ '\xF0' }; // U+1F4F7 CAMERA (4 bytes)
const std::string u8String1_2{ '\x9F' };
const std::string u8String1_3{ '\x93' };
const std::string u8String1_4{ '\xB7' };
const std::wstring u16StringComp1{
gsl::narrow_cast<wchar_t>(0xD83D), // U+1F4F7 CAMERA (surrogate pair)
gsl::narrow_cast<wchar_t>(0xDCF7)
};
til::u8state state{};
std::wstring u16Out1{};
VERIFY_SUCCEEDED(til::u8u16(u8String1_1, u16Out1, state));
VERIFY_ARE_EQUAL(L"", u16Out1); // There should be no output for the first three bytes
VERIFY_SUCCEEDED(til::u8u16(u8String1_2, u16Out1, state));
VERIFY_ARE_EQUAL(L"", u16Out1);
VERIFY_SUCCEEDED(til::u8u16(u8String1_3, u16Out1, state));
VERIFY_ARE_EQUAL(L"", u16Out1);
VERIFY_SUCCEEDED(til::u8u16(u8String1_4, u16Out1, state));
VERIFY_ARE_EQUAL(u16StringComp1, u16Out1);
}In debug builds:
StartGroup: Utf8Utf16ConvertTests::TestU8ToU16OneByOne
Verify: SUCCEEDED(til::u8u16(u8String1_1, u16Out1, state))
Verify: AreEqual(L"", u16Out1)
Error: TAEF: [HRESULT 0x800706BE] A failure occurred while running a test operation: 'Utf8Utf16ConvertTests::TestU8ToU16OneByOne'. (The process hosting the test code was unexpectedly terminated with exit code 0x00000003 while invoking a test operation.)
EndGroup: Utf8Utf16ConvertTests::TestU8ToU16OneByOne [Failed]
In release builds:
StartGroup: Utf8Utf16ConvertTests::TestU8ToU16OneByOne
Verify: SUCCEEDED(til::u8u16(u8String1_1, u16Out1, state))
Verify: AreEqual(L"", u16Out1)
Verify: SUCCEEDED(til::u8u16(u8String1_2, u16Out1, state))
Error: Verify: AreEqual(L"", u16Out1) - Values (, �) [File: E:\src\OpenConsole\src\til\ut_til\u8u16convertTests.cpp, Function: Utf8Utf16ConvertTests::TestU8ToU16OneByOne, Line: 188]
EndGroup: Utf8Utf16ConvertTests::TestU8ToU16OneByOne [Failed]
/cc @german-one