⚠️ This version introduces a breaking change in utf8_length_from_utf16_with_replacement. cc @anonrig We allow the breaking change on the assumption that nobody has had time to use our new function and if they do, the patch is simple (trivial). It is agains the practice of simdutf to introduce such breaking changes, so it is an exception.

What's Changed

• Return more information from utf8_length_from_utf16_with_replacement by @erikcorry in #860

New Contributors

• @wineee made their first contribution in #862

Full Changelog: v7.6.0...v7.7.0

What's Changed

• support reproducibility for debug sources by @hongxu-jia in #848
• Add --filter option to only run matching benchmarks by @erikcorry in #858
• UTF16 to UTF8 length with replacement by @lemire and @erikcorry in #851

New Contributors

• @davidfetter made their first contribution in #846
• @hongxu-jia made their first contribution in #848

Full Changelog: v7.5.0...v7.6.0

What's Changed

• Implement rvv validate_utf16_as_ascii function by @tantei3 in #836
• Enable SIMD generic validate_utf16_as_ascii for lsx + lasx + ppc64 by @tantei3 in #837
• Implement to_well_formed_utf16 for rvv by @tantei3 in #838
• Typo fix: parem to param by @jasseeeem in #841
• utf16fix_block_rvv: improve mask shift by @camel-cdr in #842
• converting binary data to base64 with lines by @lemire in #840

New Contributors

• @jasseeeem made their first contribution in #841

Full Changelog: v7.4.0...v7.5.0

What's Changed

• improving support for legacy GCC and validate_utf16_as_ascii by @lemire in #833 This fixes both #832 and #831

The new feature of this minor release is that we can check whether an UTF-16 string is 'ASCII' meaning that it can be converted to ASCII without any loss. This was requested by @trflynn89 of the Ladybird project.

/**
 * Validate the ASCII string as a UTF-16 sequence.
 * An UTF-16 sequence is considered an ASCII sequence
 * if it could be converted to an ASCII string losslessly.
 *
 * Overridden by each implementation.
 *
 * @param buf the UTF-16 string to validate.
 * @param len the length of the string in bytes.
 * @return true if and only if the string is valid ASCII.
 */
simdutf_warn_unused bool validate_utf16_as_ascii(const char16_t *buf,
                                                 size_t len) noexcept;

Full Changelog: v7.3.6...v7.4.0

What's Changed

• fixing issue 829 by @lemire in #830

This patch should only concern users of the trim_partial_utf16 function.

Full Changelog: v7.3.5...v7.3.6

What's Changed

• Improving the performance of simdutf::find and adding a benchmark for simdutf::find.

What's Changed

• fixing Issue 824 by @lemire in #825 We are fixing a minor issued (defined as standard compliance with TC39 base64 proposal). When using last_chunk_handling::stop_before_partial, we would sometimes we would either consume too many or too few characters, the difference being due to ignorable characters. Thus, for a string made entirely of spaces, we might report consuming no character whereas the proposed standard wants us to consume all characters. Conversely, when stopping before a partial chunk, we would consume all characters up to that point, whereas the proposed standard would want us to consume no trailing ignorable characters. Thanks to @syg for the report. Ping to @trflynn89, @Constellation, and @anonrig. We added corresponding tests.

Full Changelog: v7.3.3...v7.3.4

What's Changed

This patch release fixes minor documentation issues, an issue with UTF-8 BOM detection and an issue with stop_before_partial last chunk handling. Thanks to @syg for the report. Ping to @trflynn89, @Constellation, and @anonrig.

• more portable pic by @lemire in #816
• Minor README updates for clarity by @dzfrias in #817
• fixing issue 818 by @lemire in #819
• add qemu to test LoongArch64 by @Xiao-Tao in #820
• Issue 821 by @lemire in #823

New Contributors

• @dzfrias made their first contribution in #817

Full Changelog: v7.3.2...v7.3.3

What's Changed

• adding tests to check feature macros by @lemire in #813
• adding a nice to_well_formed benchmark by @lemire in #814

Full Changelog: v7.3.1...v7.3.2

What's Changed

• Fix macOS build by replacing --gc-sections with -dead_strip by @weqeqq in #804
• reduce the binary size under macos by @lemire in #805
• adds some documentation for std::span overloads by @lemire in #803
• Support macOS cross-compilation with Clang in osxcross by @weqeqq in #806
• adding emscripten build test by @lemire in #807
• Fix MSVC noexcept analysis warnings by @MichalPetryka in #809
• Implement lsx + lasx find function by @tantei3 in #808
• CMake: Guard shell-based tests with SIMDUTF_TESTS by @ADKaster in #811
• Implement rvv find function by @tantei3 in #810

New Contributors

• @weqeqq made their first contribution in #804
• @MichalPetryka made their first contribution in #809
• @ADKaster made their first contribution in #811

Full Changelog: v7.3.0...v7.3.1