Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@LongCatIsLooong
Copy link
Contributor

@LongCatIsLooong LongCatIsLooong commented Feb 23, 2023

I'm planning to keep Ahem the default until the framework tests are migrated (flutter/flutter#121306).

Add a new test font that provides accurate font & glyph metrics and some Chinese character codepoint coverage (not sure if that's still needed).

Ahem's 1000 upem (not a power of 2) causes FreeType to truncate numbers when computing metrics, since it will be used as a divisor and calculations are done using 26.6 fixed-point numbers. For example when point size = 1, FreeType would report an x advance of 0.99998 instead of 1. Currently we round most of the horizontal metrics so there are tons of expect(text.width, 100) in framework tests. Those tests are going to fail once the rounding is removed.
The new font covers most codepoint ranges Ahem covers, with glyphs that look almost identical to glyphs from Ahem, except with a different baseline location (ASCENT : DESCENT = 3 : 1, since Ahem's ratio ASCENT : DESCENT = 4 : 1 can't be preserved when upem is a power of 2).

Some font metrics: ascent = 768, descent = 256, units per em = 1024

Glyph Mappings by Scripts

\ Script
Glyph
DFLT grek hani latn
Square codepoint(s): 0x21-0x26, 0x28-0x40, 0x5b-0x60, 0x7b-0x7e, 0xa1-0xa9, 0xab-0xb9, 0xbb-0xbf, 0xd7, 0xf7, 0x2c6-0x2c7, 0x2c9, 0x2d8-0x2dd, 0x2013-0x2014, 0x2018-0x201a, 0x201c-0x201e, 0x2020-0x2022, 0x2026, 0x2030, 0x2039-0x203a, 0x2044, 0x2122, 0x2202, 0x2206, 0x220f, 0x2211-0x2212, 0x2219-0x221a, 0x221e, 0x222b, 0x2248, 0x2260, 0x2264-0x2265, 0x22f2, 0x25ca, 0xf000-0xf002
character(s): ! " # $ % & ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ [ \ ] ^ _ ` { | } ~ ¡ ¢ £ ¤ ¥ ¦ § ¨ © « ¬ <SOFT HYPHEN> ® ¯ ° ± ² ³ ´ µ · ¸ ¹ » ¼ ½ ¾ ¿ × ÷ ˆ ˇ ˉ ˘ ˙ ˚ ˛ ˜ ˝ <0xf000> <0xf001> <0xf002>
codepoint(s): 0x394, 0x3a5, 0x3a7, 0x3a9, 0x3bc, 0x3c0, 0x2126
character(s): Δ Υ Χ Ω μ π
codepoint(s): 0x3007, 0x4e00, 0x4e03, 0x4e09, 0x4e2d, 0x4e5d, 0x4e8c, 0x4e94, 0x516b, 0x516d, 0x5341, 0x5426, 0x56d7, 0x56db, 0x571f, 0x6587, 0x6587, 0x662f, 0x6728, 0x672c, 0x6b63, 0x6c34, 0x6d4b, 0x706b, 0x786e, 0x8bd5, 0x91d1
character(s):
codepoint(s): 0x41-0x5a, 0x61-0x7a, 0xaa, 0xba, 0xc0-0xc8, 0xca-0xd6, 0xd8-0xf6, 0xf8-0xff, 0x131, 0x152-0x153, 0x178, 0x192
character(s): A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z ª º À Á Â Ã Ä Å Æ Ç È Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ ı Œ œ Ÿ ƒ
Ascent Flushed codepoint(s): 0x70
character(s): p
Descent Flushed codepoint(s): 0xc9
character(s): É
Full Advance codepoint(s): 0x20
character(s): <SPACE>
1/2 Advance codepoint(s): 0x2002
character(s): <EN SPACE>
1/3 Advance codepoint(s): 0x2004
character(s): <THREE-PER-EM SPACE>
1/4 Advance codepoint(s): 0x2005
character(s): <FOUR-PER-EM SPACE>
1/6 Advance codepoint(s): 0x2006
character(s): <SIX-PER-EM SPACE>
1/5 Advance codepoint(s): 0x2009
character(s): <THIN SPACE>
1/10 Advance codepoint(s): 0x200a
character(s): <HAIR SPACE>
Zero Advance codepoint(s): 0xfeff
character(s): <ZERO WIDTH NO-BREAK SPACE>

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@LongCatIsLooong LongCatIsLooong changed the title [WIP] Add new test font Add new test font Feb 23, 2023
@LongCatIsLooong LongCatIsLooong marked this pull request as ready for review February 23, 2023 05:25
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@chinmaygarde
Copy link
Member

cc @Hixie

@LongCatIsLooong
Copy link
Contributor Author

(Still trying to figure out how web tests load fonts but other than that should be ready for review).

@tgucio
Copy link
Contributor

tgucio commented Feb 25, 2023

@LongCatIsLooong I'm thinking maybe adding some code points in the U+10000+ range could be useful to test behaviour with surrogate pairs? Say, Musical Symbols Range: U+1D100–U+1D1FF or Mathematical Alphanumeric Symbols Range: U+1D400–U+1D7FF.

Other interesting items for tests (thinking of some of the existing issues around complex scripts in text fields):

  • glyphs that will result in clusters in text shaping (e.g. Devanagari)
  • GSUB, GPOS features that affect caret position

@LongCatIsLooong LongCatIsLooong force-pushed the test-font branch 2 times, most recently from 5992051 to d7da67c Compare February 26, 2023 08:25
@LongCatIsLooong LongCatIsLooong force-pushed the test-font branch 3 times, most recently from 1411e77 to 4761add Compare February 26, 2023 09:55
@LongCatIsLooong
Copy link
Contributor Author

This font is meant for framework tests. In the framework we can tell whether a codepoint is in the supplementary planes by looking at the range and don't have to rely on the shaper so I think it's rarely needed to have glyph mappings for that. If you have specific needs feel free to update the script and regenerate the font (but Skia is likely a better place to write tests to verify the shapers are working as expected, the framework currently doesn't direct configure shapers AFAIK), or use FontLoader to load a suitable font.

@LongCatIsLooong LongCatIsLooong marked this pull request as draft February 28, 2023 08:56
@tgucio
Copy link
Contributor

tgucio commented Feb 28, 2023

@LongCatIsLooong yes I meant framework tests for issues like
flutter/flutter#118403 (clusters from OpenType shaping) or maybe also flutter/flutter#98516 (clusters from combining marks) where the shaper most likely works as expected but the framework doesn't handle caret location well. Still, I guess you're right this might be a topic for another PR.

For this one perhaps we could just add some > U+10000 characters to CMAP as that should be cheap.

Copy link
Contributor

@mdebbar mdebbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Web changes look good to me with some minor suggestions!

Comment on lines 77 to 85
final FontManager fontManager = _testFontManager = FontManager()
..downloadAsset(
ahemFontFamily, 'url($ahemFontUrl)', const <String, String>{})
..downloadAsset(
flutterTestFontFamily, 'url($flutterTestFontUrl)', const <String, String>{})
..downloadAsset(
robotoFontFamily, 'url($robotoTestFontUrl)', const <String, String>{})
..downloadAsset(
robotoVariableFontFamily, 'url($robotoVariableTestFontUrl)', const <String, String>{});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: we could put these fonts in a Map and loop through them instead of calling one by one. Like:

final Map<String, String> testFonts = <String, String>{
  ahemFontFamily: ahemFontUrl,
  flutterTestFontFamily: flutterTestFontUrl,
  robotoFontFamily: robotoFontUrl,
  robotoVariableFontFamily: robotoVariableFontUrl,
};

}
return fontFamily;
final String fontFamily = this.fontFamily.isEmpty ? FlutterViewEmbedder.defaultFontFamily : this.fontFamily;
// In the flutter tester environment, we use a predictable-size test fonts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// In the flutter tester environment, we use a predictable-size test fonts.
// In the flutter tester environment, we use predictable-size test fonts.

@jason-simmons
Copy link
Member

Overall approach LGTM

The tests are segfaulting on CI. In particular, the runs of flutter_tester with ASAN are reporting a null pointer dereference in flutter::GetTestFontData.

Comment on lines +150 to +151
api_file = "//flutter/third_party/web_test_fonts/lib/web_test_fonts.dart"
input_dir = "//flutter/third_party/web_test_fonts/lib/web_test_fonts/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry forgot to track those files. The tests are passing now.

@LongCatIsLooong
Copy link
Contributor Author

@tgucio I think the issues you mentioned are likely skparagraph issues and it's better to add tests to skia, as in those cases it seems the framework isn't doing anything particularly wrong, and it's going to fail faster so should there be any regressions skia maintainers will notice that without having to roll the changes into flutter. Also for skparagraph it's easy to include/load font assets in their tests so you could use existing fonts.

@LongCatIsLooong
Copy link
Contributor Author

I didn't realize we were running flutter_tester without flutter tool. I guess It's safer to embed the font data so I'll keep the existing fonts as they are and embed the new font too. It looks like asan is no longer complaining.

@LongCatIsLooong LongCatIsLooong marked this pull request as ready for review March 2, 2023 22:40
@LongCatIsLooong
Copy link
Contributor Author

Looks like the tests are passing. Could you take another look at the new approach?

Copy link
Contributor

@mdebbar mdebbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

autosubmit Merge PR when tree becomes green via auto submit App

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants