Support UTF-16BE#154
Conversation
|
I have not tested the new functions yet. I am also unsure of what change_endianness_utf16 should return. Could it just be a void function? For now, it returns something silly. |
Definitively. I think it should return |
|
@NicolasJiaxin This code looks very good to me. I just think that we ought to test it (at least minimally). |
|
I added all tests and I did have a few bugs. But it is all working now, except that the debug tests on Visual Studio are getting stuck and I don't know why. |
Let me investigate. |
|
@NicolasJiaxin When you do (for example) You will have that So you want to go through and do the following... That should solve the issue. You can also replace Note that your code is 'practically correct' but memory checkers might complain so better fix it. One problem with using std::vector is that the input is guaranteed to be zeroed. If you'd want to avoid that... You could also do the following... But I am not sure what the benefit would be. |
|
Ahh, thanks! |
|
@NicolasJiaxin Try adding some bits of documentation to the README. And then I think we will merge. |
|
Great stuff. Merging. |
Fix #3
Adds support of UTF-16BE by adding all analogous functions that we have for UTF-16LE. For example, validate_utf16 now comes in two flavours: validate_utf16le and validate_utf16be. However, the functions that implement the actual algorithms are now template-based on whether the input/output is little endian or big endian. If big endian, we swap the bytes of using a single shuffle when we load/store data. I also added a function to swap endianness called change_endianness_utf16.