Skip to content

Commit 8fb275a

Browse files
dbebawyRusino
andauthored
[web_paragraph] Map code units without a cluster to a cluster (flutter#195384)
Chrome gives some invisible characters, such as a soft hyphen (U+00AD), a zero width space (U+200B) or a right-to-left mark (U+200F), no text cluster. WebParagraph left their code units mapped to cluster 0, so a text range that includes one can map to the wrong clusters. On master: * Laying out a paragraph that ends with a soft hyphen or zero width space fails the assertion in `_TextClusterMapping.toTextRange`. * Laying out a paragraph whose only text is one of these characters throws a `StateError`. * Wrapping text where a space is followed by one of these characters, such as `'ab \u200F\u200Fcd'`, fails an assertion in `TextWrapper.breakLines`. * `getLineNumberAt` fails an assertion for a line that starts with one of these characters. * A right-to-left mark at the start of an LTR paragraph makes the first letter count twice in the line width. * A styled span that contains only one of these characters adds an empty box to `getBoxesForRange`. This maps each of those code units to the cluster before it, or to the first cluster at the start of the text, so a range that starts or ends at one of them does not take a cluster next to it. Bidi regions and span slices with no clusters add no bidi run and no block to the line, and a line's text range starts where the previous line ended and ends where the next line starts. A paragraph with no clusters is laid out like empty text, and the line breaker also finds a soft line break that lands on one of these characters. This was the first commit of flutter#195366 (soft hyphens in WebParagraph, for flutter#193506). @Rusino found three of these cases while reviewing that PR, wrote the fixes and tests in flutter#195377, and suggested landing this part on its own first. Her two commits from flutter#195386 add the rest: paragraphs with only these characters, and line breaks next to them. ## Testing `paragraph_edge_cases_test.dart` has a new group for these characters. Eleven of its tests fail on master. The two tests for a soft line break before a direction mark after a zero width space pass on master with Chrome 145 on my machine, so they may depend on the Chrome version. I ran the `chrome-dart2js-webparagraph-ui` suite locally on macOS with `felt test --gcs-prod`. Everything passes except "Paragraph with different locales for the same language" and "paint overflows" in `paragraph_test.dart`, which fail the same way on master on my machine. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [ ] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [ ] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md --------- Co-authored-by: Julia Lavrova <[email protected]>
1 parent 4a8e7d7 commit 8fb275a

3 files changed

Lines changed: 244 additions & 8 deletions

File tree

‎engine/src/flutter/lib/web_ui/lib/src/engine/web_paragraph/layout.dart‎

Lines changed: 57 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -101,11 +101,23 @@ class TextLayout {
101101
allClusters.addAll(span.extractClusters());
102102
}
103103
allClusters.sort((a, b) => a.start.compareTo(b.start));
104+
// The browser gives some invisible characters, such as a soft hyphen (U+00AD) or a zero
105+
// width space (U+200B), no cluster of their own. Map each of those code units to the
106+
// cluster before it (or the first cluster, at the start of the text), so that text ranges
107+
// that include them still map to cluster ranges in order.
108+
var unmappedStart = 0;
104109
for (var i = 0; i < allClusters.length; ++i) {
105110
final WebCluster cluster = allClusters[i];
111+
for (var j = unmappedStart; j < cluster.start; ++j) {
112+
_mapping.add(textIndex: j, clusterIndex: math.max(i - 1, 0));
113+
}
106114
for (int j = cluster.start; j < cluster.end; ++j) {
107115
_mapping.add(textIndex: j, clusterIndex: i);
108116
}
117+
unmappedStart = math.max(unmappedStart, cluster.end);
118+
}
119+
for (var j = unmappedStart; j < paragraph.text.length; ++j) {
120+
_mapping.add(textIndex: j, clusterIndex: math.max(allClusters.length - 1, 0));
109121
}
110122

111123
// One more dummy element in the end to avoid extra checks
@@ -153,6 +165,11 @@ class TextLayout {
153165
// Regions operate in text indexes, not cluster indexes (one cluster can contain several text points)
154166
// We need to convert one into another
155167
final ClusterRange clusterRange = _mapping.toClusterRange(region.start, region.end);
168+
if (clusterRange.isEmpty) {
169+
// A bidi region containing only clusterless characters (such as a standalone directional
170+
// mark or soft hyphen) owns no clusters and produces no visual run.
171+
continue;
172+
}
156173
final run = BidiRun(clusterRange, region.level);
157174
bidiRuns.add(run);
158175
}
@@ -172,7 +189,9 @@ class TextLayout {
172189
void wrapText(double width) {
173190
lines.clear();
174191

175-
if (paragraph.text.isEmpty) {
192+
// A paragraph that is empty or contains only characters without clusters has only the
193+
// trailing EmptyCluster and produces no lines.
194+
if (allClusters.length == 1) {
176195
// Empty text still has to have some metrics
177196
paragraph.width = width;
178197
paragraph.maxIntrinsicWidth = 0;
@@ -240,8 +259,16 @@ class TextLayout {
240259
);
241260
}
242261

243-
// Arrange line vertically, calculate metrics and bounds
244-
final ui.TextRange contentTextRange = _mapping.toTextRange(contentClusterRange);
262+
// Arrange line vertically, calculate metrics and bounds.
263+
// Leading clusterless characters (at offset 0 or right after a hard line break) map to the
264+
// first visible cluster after them, so start the line's text range where the previous line's
265+
// hard break ended (or at 0 for the first line) to keep line ranges contiguous.
266+
final int lineStart = lines.isEmpty ? 0 : lines.last.hardLineBreakRange.end;
267+
final ui.TextRange rawContentTextRange = _mapping.toTextRange(contentClusterRange);
268+
final contentTextRange = ui.TextRange(
269+
start: math.min(lineStart, rawContentTextRange.start),
270+
end: rawContentTextRange.end,
271+
);
245272
final ui.TextRange whitespaceTextRange = _mapping.toTextRange(whitespaceClusterRange);
246273
final ui.TextRange hardlineTextRange = _mapping.toTextRange(hardlineClusterRange);
247274
// A line includes the trailing newline if it has a hard line break that
@@ -253,7 +280,9 @@ class TextLayout {
253280
hardlineClusterRange.isNotEmpty && hardlineClusterRange.end == allClusters.length - 1;
254281
final allTextRange = ui.TextRange(
255282
start: contentTextRange.start,
256-
end: includesTrailingNewline ? hardlineTextRange.end : whitespaceTextRange.end,
283+
// Use hardlineTextRange.start rather than whitespaceTextRange.end so clusterless characters
284+
// following trailing spaces are included in this line's text range.
285+
end: includesTrailingNewline ? hardlineTextRange.end : hardlineTextRange.start,
257286
);
258287
// TODO(jlavrova): Should we use a TextLineBuilder pattern instead?
259288
final line = TextLine(
@@ -382,6 +411,11 @@ class TextLayout {
382411
bidiLineSpanTextRange.start,
383412
bidiLineSpanTextRange.end,
384413
);
414+
if (bidiLineSpanRange.isEmpty) {
415+
// A span slice containing only clusterless characters (such as a standalone U+00AD or
416+
// U+200B span) has no clusters to measure or paint on this line.
417+
continue;
418+
}
385419
final LineBlock block;
386420
final double blockWidth;
387421
if (span is PlaceholderSpan) {
@@ -569,6 +603,12 @@ class TextLayout {
569603
}
570604

571605
void _calculatePaintBounds() {
606+
if (lines.isEmpty) {
607+
// A non-empty paragraph containing only clusterless characters has no lines to paint.
608+
paragraph.paintBounds = ui.Rect.zero;
609+
return;
610+
}
611+
572612
double left = double.infinity;
573613
double top = double.infinity;
574614
double right = double.negativeInfinity;
@@ -1075,7 +1115,19 @@ class _TextClusterMapping {
10751115
final int clusterIndex = toClusterIndex(start);
10761116
return ClusterRange.collapsed(clusterIndex);
10771117
}
1078-
return ClusterRange(start: toClusterIndex(start), end: toClusterIndex(end - 1) + 1);
1118+
int startClusterIndex = toClusterIndex(start);
1119+
if (_clusters[startClusterIndex].end <= start) {
1120+
// `start` has no cluster of its own and is mapped to the cluster before it, so the range
1121+
// starts at the next cluster.
1122+
startClusterIndex += 1;
1123+
}
1124+
int endClusterIndex = toClusterIndex(end - 1) + 1;
1125+
if (_clusters[endClusterIndex - 1].start >= end) {
1126+
// The code unit at (end - 1) has no cluster of its own at the start of the text and is
1127+
// mapped to the first cluster after it, so the range ends before that cluster.
1128+
endClusterIndex -= 1;
1129+
}
1130+
return ClusterRange(start: startClusterIndex, end: endClusterIndex);
10791131
}
10801132

10811133
ui.TextRange toTextRange(ClusterRange clusterRange) {

‎engine/src/flutter/lib/web_ui/lib/src/engine/web_paragraph/wrapper.dart‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,17 @@ class TextWrapper {
3131
double get height => _height;
3232
double _height = 0.0;
3333

34-
bool _isSoftLineBreak(WebCluster cluster) {
35-
return _layout.codeUnitFlags.hasFlag(cluster.start, CodeUnitFlag.softLineBreak);
34+
bool _isSoftLineBreak(int index) {
35+
// A soft break after a clusterless character (such as U+200B or U+00AD) may land on a
36+
// subsequent clusterless code unit (such as U+200E or U+200F) before the next cluster starts.
37+
final int end = _layout.allClusters[index].start;
38+
final int start = index > 0 ? _layout.allClusters[index - 1].end : end;
39+
for (var i = start; i <= end; ++i) {
40+
if (_layout.codeUnitFlags.hasFlag(i, CodeUnitFlag.softLineBreak)) {
41+
return true;
42+
}
43+
}
44+
return false;
3645
}
3746

3847
bool _isHardLineBreak(WebCluster cluster) {
@@ -71,7 +80,7 @@ class TextWrapper {
7180
} else {
7281
continue;
7382
}
74-
} else if (_isSoftLineBreak(cluster) && line.isNotEmpty) {
83+
} else if (_isSoftLineBreak(index) && line.isNotEmpty) {
7584
// Mark the potential line break and then continue with the current cluster as usual
7685
if (line.hasLeadingWhitespaces) {
7786
// There is one case when we have to ignore this soft line break: if we only had whitespaces so far -

‎engine/src/flutter/lib/web_ui/test/webparagraph/paragraph_edge_cases_test.dart‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -833,4 +833,179 @@ Future<void> testMain() async {
833833
},
834834
);
835835
});
836+
837+
group('Paragraph characters without a cluster', () {
838+
// The browser gives some invisible characters no cluster of their own.
839+
for (final (String name, String text) in <(String, String)>[
840+
('soft hyphen', 'ab cd\u00AD'),
841+
('zero width space', 'ab cd\u200B'),
842+
]) {
843+
test('Layout handles a trailing $name', () {
844+
final builder = ui.ParagraphBuilder(ahemStyle);
845+
builder.addText(text);
846+
final ui.Paragraph paragraph = builder.build();
847+
848+
expect(() => paragraph.layout(const ui.ParagraphConstraints(width: 40)), returnsNormally);
849+
expect(paragraph.numberOfLines, 2);
850+
});
851+
}
852+
853+
for (final (String name, String character) in <(String, String)>[
854+
('soft hyphen', '\u00AD'),
855+
('zero width space', '\u200B'),
856+
]) {
857+
test('A span that starts with a $name covers only its own text', () {
858+
final builder = ui.ParagraphBuilder(ahemStyle);
859+
builder.addText('abcd');
860+
builder.pushStyle(ui.TextStyle(color: const ui.Color(0xFF0000FF)));
861+
builder.addText('${character}ef');
862+
builder.pop();
863+
final ui.Paragraph paragraph = builder.build();
864+
paragraph.layout(const ui.ParagraphConstraints(width: 1000));
865+
866+
final ui.TextBox abcd = paragraph.getBoxesForRange(0, 4).single;
867+
final List<ui.TextBox> ef = paragraph.getBoxesForRange(5, 7);
868+
expect(ef, hasLength(1));
869+
expect(ef.single.left, closeTo(abcd.right, 0.01));
870+
});
871+
872+
test('getLineNumberAt maps a leading $name to its line', () {
873+
// When a line starts with a character that has no cluster of its own (at offset 0 or
874+
// right after a hard newline), the first visible cluster starts after that character,
875+
// but the line's text range must still start at the beginning of the line so binary
876+
// search in getLineNumberAt finds it.
877+
final builder = ui.ParagraphBuilder(ahemStyle)..addText('${character}abcd\n${character}ef');
878+
final ui.Paragraph paragraph = builder.build()
879+
..layout(const ui.ParagraphConstraints(width: 1000));
880+
881+
expect(paragraph.getLineNumberAt(0), 0);
882+
expect(paragraph.getLineNumberAt(6), 1);
883+
});
884+
885+
test('A styled span containing only a $name adds no visual block', () {
886+
// A span whose text has no clusters of its own maps to an empty cluster range and must
887+
// not emit a zero-width TextBlock in visualBlocks or split selection boxes.
888+
final builder = ui.ParagraphBuilder(ahemStyle);
889+
builder.addText('ab');
890+
builder.pushStyle(ui.TextStyle(color: const ui.Color(0xFF0000FF)));
891+
builder.addText(character);
892+
builder.pop();
893+
builder.addText('cd');
894+
final ui.Paragraph paragraph = builder.build()
895+
..layout(const ui.ParagraphConstraints(width: 1000));
896+
897+
expect(paragraph.getBoxesForRange(0, 5), hasLength(2));
898+
});
899+
}
900+
901+
test('Directional marks without a cluster do not create empty or duplicate bidi runs', () {
902+
// U+200F (RLM) has no cluster of its own. At the start of an LTR paragraph it must map to
903+
// an empty cluster range [0, 0), and before a newline its empty BidiRun must be skipped
904+
// during line layout.
905+
ui.Paragraph layoutText(String text) {
906+
final builder = ui.ParagraphBuilder(ahemStyle)..addText(text);
907+
return builder.build()..layout(const ui.ParagraphConstraints(width: 1000));
908+
}
909+
910+
final ui.Paragraph abc = layoutText('abc');
911+
final ui.Paragraph leadingRlm = layoutText('\u200Fabc');
912+
expect(
913+
leadingRlm.computeLineMetrics().single.width,
914+
closeTo(abc.computeLineMetrics().single.width, 0.01),
915+
);
916+
expect(leadingRlm.getBoxesForRange(1, 4), hasLength(1));
917+
918+
final ui.Paragraph interiorRlm = layoutText('a\u200F\nb');
919+
expect(interiorRlm.numberOfLines, 2);
920+
});
921+
922+
for (final (String name, String character) in <(String, String)>[
923+
('soft hyphen', '\u00AD'),
924+
('zero width space', '\u200B'),
925+
('right-to-left mark', '\u200F'),
926+
]) {
927+
test('Layout handles a paragraph containing only a $name', () {
928+
// A paragraph whose text consists solely of characters without clusters produces only the
929+
// trailing EmptyCluster and must lay out with empty-paragraph metrics instead of throwing
930+
// on lines.first.
931+
final emptyBuilder = ui.ParagraphBuilder(ahemStyle);
932+
final ui.Paragraph emptyParagraph = emptyBuilder.build()
933+
..layout(const ui.ParagraphConstraints(width: 100));
934+
935+
final builder = ui.ParagraphBuilder(ahemStyle)..addText(character);
936+
final ui.Paragraph paragraph = builder.build();
937+
938+
expect(() => paragraph.layout(const ui.ParagraphConstraints(width: 100)), returnsNormally);
939+
expect(paragraph.numberOfLines, 0);
940+
expect(paragraph.height, emptyParagraph.height);
941+
expect(paragraph.alphabeticBaseline, emptyParagraph.alphabeticBaseline);
942+
expect(paragraph.ideographicBaseline, emptyParagraph.ideographicBaseline);
943+
expect(paragraph.maxIntrinsicWidth, 0);
944+
expect(paragraph.minIntrinsicWidth, 0);
945+
expect(paragraph.getPositionForOffset(ui.Offset.zero), const ui.TextPosition(offset: 0));
946+
expect(paragraph.getLineNumberAt(0), isNull);
947+
expect(paragraph.getGlyphInfoAt(0), isNull);
948+
});
949+
}
950+
951+
for (final (String name, String character) in <(String, String)>[
952+
('left-to-right mark', '\u200E'),
953+
('right-to-left mark', '\u200F'),
954+
]) {
955+
test('Soft line break before a $name after a zero width space wraps the line', () {
956+
// Depending on the browser's line breaker, the soft break after U+200B (LB8) can land on
957+
// the following directional mark, which has no cluster of its own. Line wrapping must
958+
// still detect the soft break before the next visible cluster.
959+
final builder = ui.ParagraphBuilder(ahemStyle)..addText('ab\u200B${character}cd');
960+
final ui.Paragraph paragraph = builder.build()
961+
..layout(const ui.ParagraphConstraints(width: 30));
962+
963+
expect(paragraph.numberOfLines, 2);
964+
expect(
965+
paragraph.getLineBoundary(const ui.TextPosition(offset: 0)),
966+
const ui.TextRange(start: 0, end: 4),
967+
);
968+
expect(
969+
paragraph.getLineBoundary(const ui.TextPosition(offset: 4)),
970+
const ui.TextRange(start: 4, end: 6),
971+
);
972+
});
973+
}
974+
975+
test('Soft line break and getLineBoundary handle clusterless characters after spaces', () {
976+
// When trailing spaces are followed by characters that have no cluster of their own, soft
977+
// wrapping must still detect the break after the spaces and the line's text range must cover
978+
// the unmapped code units.
979+
final wrappedBuilder = ui.ParagraphBuilder(ahemStyle)..addText('ab \u200F\u200Fcd');
980+
final ui.Paragraph wrapped = wrappedBuilder.build()
981+
..layout(const ui.ParagraphConstraints(width: 30));
982+
983+
expect(wrapped.numberOfLines, 2);
984+
expect(wrapped.getLineNumberAt(3), 0);
985+
expect(wrapped.getLineNumberAt(4), 0);
986+
expect(
987+
wrapped.getLineBoundary(const ui.TextPosition(offset: 3)),
988+
const ui.TextRange(start: 0, end: 5),
989+
);
990+
expect(
991+
wrapped.getLineBoundary(const ui.TextPosition(offset: 4)),
992+
const ui.TextRange(start: 0, end: 5),
993+
);
994+
expect(
995+
wrapped.getLineBoundary(const ui.TextPosition(offset: 5)),
996+
const ui.TextRange(start: 5, end: 7),
997+
);
998+
999+
final trailingBuilder = ui.ParagraphBuilder(ahemStyle)..addText('ab \u200B');
1000+
final ui.Paragraph trailing = trailingBuilder.build()
1001+
..layout(const ui.ParagraphConstraints(width: 100));
1002+
1003+
expect(trailing.numberOfLines, 1);
1004+
expect(trailing.getLineNumberAt(3), 0);
1005+
expect(
1006+
trailing.getLineBoundary(const ui.TextPosition(offset: 3)),
1007+
const ui.TextRange(start: 0, end: 4),
1008+
);
1009+
});
1010+
});
8361011
}

0 commit comments

Comments
 (0)