Skip to content

About Full Unicode (Plane 1-16) support  #2538

@cloudwu

Description

@cloudwu

ImWchar is defined as unsigned short (16bit int) now.

https://github.com/ocornut/imgui/blob/master/imgui.h#L124

typedef unsigned short ImWchar;     // A single U16 character for keyboard input/display. We encode them as multi bytes UTF-8 when used in strings.

And AddInputCharactersUTF8 / ImTextStrFromUtf8 / ImTextCountCharsFromUtf8 discard the code points in Plane 1-16 ,

https://github.com/ocornut/imgui/blob/master/imgui.cpp#L1263
https://github.com/ocornut/imgui/blob/master/imgui.cpp#L1678-L1710

void ImGuiIO::AddInputCharactersUTF8(const char* utf8_chars)
{
    while (*utf8_chars != 0)
    {
        unsigned int c = 0;
        utf8_chars += ImTextCharFromUtf8(&c, utf8_chars, NULL);
        if (c > 0 && c <= 0xFFFF)    // <---- DISCARDS c >= 0x10000
            InputQueueCharacters.push_back((ImWchar)c);
    }
}
       if (c < 0x10000)    // FIXME: Losing characters that don't fit in 2 bytes
            *buf_out++ = (ImWchar)c;

ImTextCharToUtf8 can not convert the code points great than 0x10000 , for example , U+20628 (𠘨).

https://github.com/ocornut/imgui/blob/master/imgui.cpp#L1713

    //else if (c < 0x10000)
    {
        if (buf_size < 3) return 0;
        buf[0] = (char)(0xe0 + (c >> 12));
        buf[1] = (char)(0x80 + ((c>> 6) & 0x3f));
        buf[2] = (char)(0x80 + ((c ) & 0x3f));
        return 3;
    }

I think full unciode support is not complicated, all we need is define ImWchar as unsigned int and removing the branch (c < 0x10000).

Is there any other reason to use u16 internal rather than u32 ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions