Is there any performance concern with nextCharacter and previousCharacter? They iterate the entire string in order to avoid bugs where the extent is in the middle of an extended grapheme cluster. If we can assume that won't happen, or we are ok with these methods not finding the end/beginning of the grapheme cluster when it does happen, then we can optimize by starting our search from extent.

If we kept track a flag around that indicated if the string contained anything aside from UTF-16 characters, then we could opt to use simple algorithms.

For strings that contained surrogate pairs or extended grapheme clusters: if we could scan backwards from an arbitrary index and discover if we were in the middle of such a character, then finding the next character (or previous character) index would be cheap. So long as there was a small limit to how far we had to scan backwards.

I wrote a proof-of-concept and it seemed like something like this would work and would be potentially much faster. I'll follow up after this is merged.

It's not possible to improve performance using the characters package as expected. It can find characters by iterating from the start or end of a string, but at some offset into a string, it doesn't know where the character boundaries are. Performance is likely to be worse iterating from the start/end of a long string rather than using the existing logarithmic algorithm.

See below

Is this really sufficient to detect a half of a UTF-16 surrogate pair? According to Wikipedia, the first byte of surrogate pair values fall into one of two ranges:

high surrogates (0xD800–0xDBFF), low surrogates (0xDC00–0xDFFF

Not sure if this test is sufficient? Note also: I realize that this code hasn't really changed.

I've opened a separate issue for this and assigned myself. I'll look into it and merge a small PR separately if it is indeed wrong. #59513

This code does detect surrogates values among UTF-16 code units.

The documentation talks about "two or more UTF-16 codeunits", which is confusing because glyphs can indeed be comprised of more than one code point, but that's unrelated to surrogates.
I'd say that the documentation is almost entirely wrong, but the function does what the name says it does.

Thanks for the clarification. I'll update the documentation on #59513.

Presumably string.length is assumed to be beyond the end of the string, even though string.characters.length might have a smaller value than string.length: what are callers supposed to do with the return value here? In other words, what return-value is expected if the next character past extent is beyond the end of the string?

We should have tests for nextCharacter(s.length, s), nextCharacter(s.length -1, s), previousCharacter(0, s), previousCharacter(s.length, s). Or maybe invalid extents should cause an assert?

I've made this more straightforward in b3cdde2.

1. I assert that the input must be between 0 and string.length, inclusive.
2. Asking for the previous character from index 0 returns 0. Asking for the next character after string.length returns string.length.
3. I clarified that the input and output of this method are all positions in the string (not character indices).
4. I tested all missing edge cases, including cases that assert.

If we kept track a flag around that indicated if the string contained anything aside from UTF-16 characters, then we could opt to use simple algorithms.

For strings that contained surrogate pairs or extended grapheme clusters: if we could scan backwards from an arbitrary index and discover if we were in the middle of such a character, then finding the next character (or previous character) index would be cheap. So long as there was a small limit to how far we had to scan backwards.

See below

This method and the nextCharacter method should make meaning of their extent parameter and return value really clear, i.e. are they string indices or string.character indices. They should make how they handle edge cases clear as well.

That line is equivalent to:

   return _isWhitespace(currentString.codeUnitAt(0));

Ah thanks, good call!

This will be fixed when I reland this PR in #59778.