Add support for more accents in mathtext - #23189

The remaining errors after removing the single letter cases above (keeping H) are:

Now:

Earlier:

so a consequence of the actual characters being used.

Now:

Earlier:

The addition of check leads to that the checkmark is used here. I do not really understand this test, nor the use of _accentprefixed.

Now:

Earlier:

So a consequence of the combined character not being in the font.

For the first, and I assume the second, case, the right thing would be to update the images.

For the final case, there should be some checking if the glyph exists in the used font.

accentprefixed is being handled (removed) at #22950.

@anntzer Do you know if #22950 will enable using single character accents (that is also a starting character of another LaTeX symbol)?

Also, do you have any idea how one can detect if a glyph actually exists as in the ṡ turning into ¤ in the image above? (I do not think it is Matplotlib that does that substitution?)

first point: yes, I think that should work.
second point: I think the relevant parts are around

Thanks! Ahh, I knew I had seen that somewhere! Grepped for ¤ though...

I'm wondering if one should introduce some rcParam for the replacement. If I understand it correctly, it may not be possible for the parser to actually know the exact font being used? (Only like 'rm')

Edit: Inkscape was not in the path due to a reinstall...

Also, it seems like the svg output actually handles ṡ, but not the pdf or png output. Checking the source, it seems like something converts the combined character back into a combined accent and character. Not sure what though.

Anyway, I am wondering if one possible should try and decompose the characters once the _get_glyph-operation fails?

Example: (not relevant anymore, but may still be of interest)

import unicodedata
accent = chr(775)
withcombiningaccent = 's' + chr(775)
print(withcombiningaccent , len(withcombiningaccent))
combined = unicodedata.normalize('NFC',  withcombiningaccent)
print(combined, len(combined))
print(ord(combined))

This shows that it correctly finds https://www.codetable.net/decimal/7777

One can do unicodedata.normalize('NFD', chr(7777)) to get the two characters back again.

However, in the svg output

I also replaced some of the accents with the "proper" combining accent. So this breaks another test. But avoids having to resize \circ.

Perhaps split out the addition of new accents as a separate PR, which should be fairly uncontroversial?

I suspect that general handling of combining characters would basically require harfbuzz (which knows how to position an accent by itself, e.g. the classic "zalgo" text h̷̡̦͚́͛̅̔̅̊͘ě̶͚̣̭́̉͜ļ̴͚͙̝̑̒l̸̛̙̹ͅơ̵͎̻͔̯̊ ̶̨̨͖̥̺͓̽̋̒͝w̶̨̗̻̥̜͍̮̏͛͒͝o̷̟͆̍̓̚ŗ̵̢͔̦̑͗̑̑̃l̸̲̥̲̹͖̔̇̾̏͆d̴͍̲̓̄̑̉̌̇͜) + switching from bakoma to lm-math, to have access to the combining characters... Still,

If I understand it correctly, it may not be possible for the parser to actually know the exact font being used?

I think that's actually possible? e.g. Char._update_metrics does self._metrics = self.font_output.get_metrics(self.font, self.font_class, self.c, self.fontsize, self.dpi) loads the metrics of a glyph in the current concrete font, so that can certainly check whether the glyph exists in that font.

You are correct that it was possible. I couldn't follow the order of things happening properly.

I think that the zalgo support is actually not that much affected by this. It is just that when there are proper glyphs available these will be used, if not, it will be as before (which I guess supported zalgo to some extent). See for example the test with r'$\mathring{A} \AA$', where now both characters render identical (Å). (This is a rather good test for this feature, possibly including a few more Unicode characters.)

One may even consider consider checking if a Unicode character can be split.

Anyway, this should really wait until #22950 is merged so that more accents can be added. One could also consider adding support for other combining accents, like cedilla and ogonek, which at least should work when there are available combined characters. Maybe one should have two separate groups of accents: the current ones where it is possible to "create" decently looking combinations and those like cedilla and ogonek which may have a valid combined glyph. If those doesn't work one could error if they do not combine or the glyph is not available.

(I tried out to get combining accents below working, but I had some issues with aligning them correctly, especially since cedilla and ogonek should be without a gap and I didn't get that to work for e.g. p, which probably noone wants, but still...)

There are now some more things changed:

• macron and overline are different
• if possible, a dotless i is used (as LaTeX does nowdays)
• there a number of new test images, primarily for illustration, as I expect them to change (note that \check is not working)

Turns out that for some characters caron (\check) is written like that https://www.compart.com/en/unicode/U+0165

Since this Pull Request has not been updated in 60 days, it has been marked "inactive." This does not mean that it will be closed, though it may be moved to a "Draft" state. This helps maintainers prioritize their reviewing efforts. You can pick the PR back up anytime - please ping us if you need a review or guidance to move the PR forward! If you do not plan on continuing the work, please let us know so that we can either find someone to take the PR over, or close it.

Likely to be revived, perhaps in a slightly different form, once raqm/harfbuzz lands.