I like that solution better than the current workarounds. I just checked, the new lexer doesn't appear in the documentation, so that's good. I don't mind shipping a lexer rewrite given it passes all existing tests, and the current test coverage is better than what we shipped the last major update with (hence the regressions!) Unless @birkenfeld has any reservations, my vote is on merging this and getting a 2.15.1 release out of the door as soon as we can.

I'm on holiday, my only reservations are in hotels. Go ahead :)

@jeanas Good to go or do you want to do anything else before merging? It's currently in draft so I cannot trivially merge this.

Coming back to this, I didn't really find value in making more commits. Let's merge it.

I agree on releasing a bugfix version soon. Maybe also include #2404 in it? (It fixes a case of catastrophic backtracking.)

Thanks! Yes -- that's enough for a point release. I'll get #2404 included as well. I should get to it tomorrow. Good work on both!