Skip to content

Commit c664bdd

Browse files
fix: prefer pointer events over semantics tap for mouse clicks on web (flutter#191529)
This PR fixes `Tap*Details` reporting the center of the semantics node and `PointerDeviceKind.unknown` for mouse clicks on web when semantics is enabled. When semantics is enabled, `ClickDebouncer` withholds the pointer events that lead up to a DOM "click" on a tappable semantics node and forwards the click as a `SemanticsAction.tap` instead. That action carries neither a position nor a device kind, so the framework synthesizes the tap at the center of the semantics node with `PointerDeviceKind.unknown`. For a merged semantics node that also means the tap is attributed to the merged node rather than to whatever was actually under the cursor. The debouncer exists for assistive technologies, which drive taps either through clicks that come with no pointer events at all, or through touch. Neither of those paths is affected by this change: when the withheld pointer events came from a mouse, they are now flushed to the framework and the click is dropped, so mouse gestures are recognized exactly like they are when semantics is off. Fixes: flutter#188859 Fixes: flutter#146063 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md --------- Co-authored-by: zhongliugo <[email protected]>
1 parent d811479 commit c664bdd

2 files changed

Lines changed: 115 additions & 4 deletions

File tree

‎engine/src/flutter/lib/web_ui/lib/src/engine/pointer_binding.dart‎

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,10 @@ typedef DebounceState = ({
212212
/// time the click event arrives, the queue was flushed (but no more than 50ms
213213
/// ago), then the click event is dropped instead under the assumption that
214214
/// the flushed pointer events are interpreted by the framework as the desired
215-
/// gesture.
215+
/// gesture. Clicks generated by a physical mouse are an exception: the queued
216+
/// pointer events are flushed and the click is dropped, because pointer
217+
/// events describe the interaction more precisely than a
218+
/// `SemanticsAction.tap` can.
216219
///
217220
/// This mechanism is in place to deal with https://github.com/flutter/flutter/issues/130162.
218221
class ClickDebouncer {
@@ -245,6 +248,26 @@ class ClickDebouncer {
245248
bool get isDebouncing => _isDebouncing;
246249
bool _isDebouncing = false;
247250

251+
/// Whether the pointer events withheld from the framework were generated by
252+
/// a physical mouse.
253+
///
254+
/// Chrome turns screen reader activation into a synthetic `pointerdown`,
255+
/// `pointerup` and `click` with `pointerType: 'mouse'`, all dispatched in a
256+
/// single task. A physical mouse click spans at least two tasks, one for the
257+
/// press and one for the release, by which point debouncing has started.
258+
bool get _isDebouncingMouseEvents {
259+
final DebounceState? state = _state;
260+
if (state == null || !state.started) {
261+
return false;
262+
}
263+
final List<QueuedEvent> queue = state.queue;
264+
if (queue.isEmpty) {
265+
return false;
266+
}
267+
final List<ui.PointerData> data = queue.first.data;
268+
return data.isNotEmpty && data.first.kind == ui.PointerDeviceKind.mouse;
269+
}
270+
248271
/// Processes a pointer event.
249272
///
250273
/// If semantics are off, simply forwards the event to the framework.
@@ -301,6 +324,24 @@ class ClickDebouncer {
301324
}
302325

303326
if (isListening) {
327+
if (_isDebouncingMouseEvents) {
328+
// The queued pointer events carry the exact position and device kind of
329+
// the interaction, while `SemanticsAction.tap` carries neither, forcing
330+
// the framework to synthesize a tap at the center of the semantics
331+
// node. Flush the pointer events so that mouse gestures are recognized
332+
// exactly like they are when semantics is off.
333+
//
334+
// Assistive technologies drive taps through clicks that come with no
335+
// pointer events at all, through touch, or through mouse events
336+
// synthesized in the same task as the click, so they keep using the
337+
// semantics action below.
338+
//
339+
// See https://github.com/flutter/flutter/issues/188859
340+
click.stopPropagation();
341+
_flush();
342+
return;
343+
}
344+
304345
// There's a pending queue of pointer events. Prefer sending the tap action
305346
// instead of pointer events, because the pointer events may not land on the
306347
// combined semantic node and miss the click/tap.

‎engine/src/flutter/lib/web_ui/test/engine/pointer_binding_test.dart‎

Lines changed: 73 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3433,6 +3433,13 @@ void _testClickDebouncer({required PointerBinding Function() getBinding}) {
34333433
return Future.delayed(Duration.zero);
34343434
}
34353435

3436+
// Mouse clicks are forwarded to the framework as pointer events rather than
3437+
// as `SemanticsAction.tap`, so tests that exercise the semantics action use
3438+
// touch, like an assistive technology would.
3439+
DomEvent touchDown() =>
3440+
context.multiTouchDown(const <_TouchDetails>[_TouchDetails(pointer: 1)])[0];
3441+
DomEvent touchUp() => context.multiTouchUp(const <_TouchDetails>[_TouchDetails(pointer: 1)])[0];
3442+
34363443
void testWithSemantics(
34373444
String description,
34383445
Future<void> Function() body, {
@@ -3546,7 +3553,7 @@ void _testClickDebouncer({required PointerBinding Function() getBinding}) {
35463553
testElement.setAttribute('flt-tappable', '');
35473554
view.dom.semanticsHost.appendChild(testElement);
35483555

3549-
testElement.dispatchEvent(context.primaryDown());
3556+
testElement.dispatchEvent(touchDown());
35503557
// ClickDebouncer does not start debouncing right away.
35513558
expect(PointerBinding.clickDebouncer.isDebouncing, isTrue);
35523559
expect(PointerBinding.clickDebouncer.debugState, isNotNull);
@@ -3576,13 +3583,13 @@ void _testClickDebouncer({required PointerBinding Function() getBinding}) {
35763583
view.dom.semanticsHost.appendChild(testElement);
35773584

35783585
// A `pointerdown` kicks off the debouncing process.
3579-
testElement.dispatchEvent(context.primaryDown());
3586+
testElement.dispatchEvent(touchDown());
35803587
expect(PointerBinding.clickDebouncer.isDebouncing, isTrue);
35813588
expect(PointerBinding.clickDebouncer.debugState, isNotNull);
35823589
expect(PointerBinding.clickDebouncer.debugState!.queue, hasLength(1));
35833590

35843591
// A `pointerup` in the same event loop should not throw.
3585-
expect(() => testElement.dispatchEvent(context.primaryUp()), returnsNormally);
3592+
expect(() => testElement.dispatchEvent(touchUp()), returnsNormally);
35863593
expect(PointerBinding.clickDebouncer.isDebouncing, isTrue);
35873594
expect(PointerBinding.clickDebouncer.debugState, isNotNull);
35883595
expect(PointerBinding.clickDebouncer.debugState!.queue, hasLength(2));
@@ -3718,19 +3725,82 @@ void _testClickDebouncer({required PointerBinding Function() getBinding}) {
37183725
testWithSemantics('Forwards click to framework when debouncing and listening', () async {
37193726
expect(PointerBinding.clickDebouncer.isDebouncing, false);
37203727

3728+
final DomElement testElement = createDomElement('flt-semantics');
3729+
testElement.setAttribute('flt-tappable', '');
3730+
view.dom.semanticsHost.appendChild(testElement);
3731+
testElement.dispatchEvent(touchDown());
3732+
expect(PointerBinding.clickDebouncer.isDebouncing, true);
3733+
3734+
await nextEventLoop();
3735+
final DomEvent click = createDomMouseEvent('click', <Object?, Object?>{
3736+
'clientX': testElement.getBoundingClientRect().x,
3737+
'clientY': testElement.getBoundingClientRect().y,
3738+
});
3739+
3740+
PointerBinding.clickDebouncer.onClick(click, view.viewId, 42, true);
3741+
expect(pointerPackets, isEmpty);
3742+
expect(semanticsActions, <CapturedSemanticsEvent>[(type: ui.SemanticsAction.tap, nodeId: 42)]);
3743+
});
3744+
3745+
// Regression test for https://github.com/flutter/flutter/issues/188859
3746+
//
3747+
// A `SemanticsAction.tap` carries no position and no device kind, so the
3748+
// framework has to synthesize the tap at the center of the semantics node.
3749+
// Mouse clicks come with pointer events that describe the interaction
3750+
// precisely, so those are flushed to the framework instead.
3751+
testWithSemantics('Flushes pointer events instead of tap action for mouse clicks', () async {
3752+
expect(PointerBinding.clickDebouncer.isDebouncing, false);
3753+
37213754
final DomElement testElement = createDomElement('flt-semantics');
37223755
testElement.setAttribute('flt-tappable', '');
37233756
view.dom.semanticsHost.appendChild(testElement);
37243757
testElement.dispatchEvent(context.primaryDown());
37253758
expect(PointerBinding.clickDebouncer.isDebouncing, true);
37263759

37273760
await nextEventLoop();
3761+
testElement.dispatchEvent(context.primaryUp());
3762+
expect(
3763+
reason: 'Events are withheld from the framework while debouncing',
3764+
pointerPackets,
3765+
isEmpty,
3766+
);
3767+
37283768
final DomEvent click = createDomMouseEvent('click', <Object?, Object?>{
37293769
'clientX': testElement.getBoundingClientRect().x,
37303770
'clientY': testElement.getBoundingClientRect().y,
37313771
});
3772+
PointerBinding.clickDebouncer.onClick(click, view.viewId, 42, true);
3773+
3774+
expect(PointerBinding.clickDebouncer.isDebouncing, false);
3775+
expect(pointerPackets, <ui.PointerChange>[
3776+
ui.PointerChange.add,
3777+
ui.PointerChange.down,
3778+
ui.PointerChange.up,
3779+
]);
3780+
expect(semanticsActions, isEmpty);
3781+
});
3782+
3783+
// Chrome turns screen reader activation into mouse pointer events and a click
3784+
// at the center of the element, all dispatched in a single task. Those must
3785+
// keep producing `SemanticsAction.tap`, otherwise nodes that only handle the
3786+
// semantics action, such as `Semantics(onTap: ...)`, stop responding.
3787+
testWithSemantics('Forwards tap action for mouse clicks within a single task', () async {
3788+
expect(PointerBinding.clickDebouncer.isDebouncing, false);
37323789

3790+
final DomElement testElement = createDomElement('flt-semantics');
3791+
testElement.setAttribute('flt-tappable', '');
3792+
view.dom.semanticsHost.appendChild(testElement);
3793+
testElement.dispatchEvent(context.primaryDown());
3794+
testElement.dispatchEvent(context.primaryUp());
3795+
expect(PointerBinding.clickDebouncer.isDebouncing, true);
3796+
3797+
final DomEvent click = createDomMouseEvent('click', <Object?, Object?>{
3798+
'clientX': testElement.getBoundingClientRect().x,
3799+
'clientY': testElement.getBoundingClientRect().y,
3800+
});
37333801
PointerBinding.clickDebouncer.onClick(click, view.viewId, 42, true);
3802+
3803+
expect(PointerBinding.clickDebouncer.isDebouncing, false);
37343804
expect(pointerPackets, isEmpty);
37353805
expect(semanticsActions, <CapturedSemanticsEvent>[(type: ui.SemanticsAction.tap, nodeId: 42)]);
37363806
});

0 commit comments

Comments
 (0)