Repository navigation
Conversation
This fixes issue flutter#188859 where ensureSemantics() breaks Tap*Details from GestureDetector on mouse click on the web. The ClickDebouncer was capturing physical mouse pointer events and synthesizing SemanticsAction.tap. The debouncer should only handle gestures on semantics tree nodes from screen readers (which dispatch synthetic pointer Type 'touch' / undefined, not 'mouse'). Now it skips debouncing when pointerType == 'mouse'.
There was a problem hiding this comment.
Code Review
This pull request introduces touchDown and touchUp helper methods in pointer_binding_test.dart to simulate touch events. It refactors existing tests to use these touch helpers instead of mouse-simulating primary events, and adds a new test specifically verifying that mouse events are forwarded to the framework when semantics is enabled. There are no review comments, and I have no feedback to provide.
When semantics is enabled, `ClickDebouncer` withholds pointer events that land on an `flt-tappable` node and, on the subsequent DOM "click", synthesizes a generic `SemanticsAction.tap` in their place. The framework turns that into a tap at the center of the widget with `PointerDeviceKind.unknown`, so a `GestureDetector` under a physical mouse cursor reports `Tap*Details` with the wrong position and kind. Forward physical mouse input straight to the framework instead. The subsequent "click" is then deduped by `_shouldSendClickEventToFramework`, because the forwarded `pointerup` sets `_lastSentPointerUpTimeStamp`. Trade-off, documented at the call site: mouse input can no longer reach `SemanticsAction.tap` by any route, so a semantics node exposing a tap action with no hit-testable gesture recognizer under the cursor stops responding to mouse clicks. Only `mouse` is exempted; `pen` and `touch` still debounce. Also removes a 200ms delay on every mouse press over a tappable node. Tests: the existing `_testClickDebouncer` cases were written against `context.primaryDown()`, which is `pointerType: 'mouse'`, so they now exercise touch instead via local `touchDown`/`touchUp` helpers. Adds a regression test that mouse input reaches the framework as pointer packets and that the following "click" does not also fire a semantics tap. Fixes flutter#188859
|
Moving this to draft — I found a problem with my own approach and I don't think it should land as written. Flagging it here because the @yjbanov @mdebbar — you own this code path, and I'd value your read on the design question rather than guessing. Short version: filtering the debouncer on
initializer->setPointerId(PointerEventFactory::kReservedNonPointerId);
if (creation_scope == SimulatedClickCreationScope::kFromAccessibility) {
initializer->setPointerId(PointerEventFactory::kMouseId);
initializer->setPointerType(pointer_type_names::kMouse);
initializer->setIsPrimary(true);
}It deliberately overrides the reserved non-pointer id with the real mouse id, sets coordinates to the element's border-box center, and Safari is unaffected, for a different reason: Incidentally, this identifies the sequence in #174046 as the Blink path — that observation was on Chrome, not Safari. Consequence of this PR on Chrome: a screen-reader activation takes the new early return, the following Where I got stuck: I initially thought the safe fallback was to keep debouncing and just flush the queued pointer events on The underlying tension seems to be that the engine can't know whether the framework's gesture arena consumed the forwarded pointer events, so any engine-only fix has to pick between coordinate fidelity and the semantics tap. Resolving it properly might need the framework to tell the engine whether a node has a real hit-testable recognizer versus a semantics-only tap action — which is well beyond this PR. Is that trade acceptable (is recognizer-less Context on the bug itself: #188859, and #146063 / #177314 look like the same root cause. |
|
This pull request has been changed to a draft. The currently pending flutter-gold status will not be able to resolve until a new commit is pushed or the change is marked ready for review again. For more guidance, visit Writing a golden file test for Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. |
| /// Physical mouse input is exempt from all of the above and is forwarded to | ||
| /// the framework directly, because a synthesized `SemanticsAction.tap` loses | ||
| /// the pointer position and device kind that a mouse gesture carries. See | ||
| /// [_maybeStartDebouncing]. |
There was a problem hiding this comment.
Restoring the pointer position part makes sense. How do we know this does not reintroduce the double-click issue, when SemanticsAction.tap and pointer down + up are both forwarded to the tap handler? Can we instead find a way to attach the pointer position to SemanticsAction.tap?
|
Closing unmerged — the |
Fixes #188859
Problem
When semantics is enabled,
Tappablemarks every tappable node withflt-tappable.ClickDebouncerthen withholds pointer events that land on such a node and, when the DOMclickarrives, synthesizes a genericSemanticsAction.tapin their place and drops the queued pointer events.The framework turns that action into a tap at the center of the widget with
PointerDeviceKind.unknown. So withensureSemantics()active, aGestureDetectorunder a physical mouse cursor getsTap*Detailswith the wrong position and the wrong device kind.Fix
Forward physical mouse input straight to the framework rather than debouncing it. The DOM
clickthat follows is then deduped by the existing_shouldSendClickEventToFrameworkcheck, because the forwardedpointerupsets_lastSentPointerUpTimeStamp. No double-fire.This also removes the 200ms press latency that mouse clicks on a tappable node currently incur.
Putting this up front because it is the thing that needs a decision before this can land. I went looking for what
pointerTypeassistive technologies actually report. The two engines differ, and Blink is the problem:Blink / Chrome —
event_dispatcher.ccdispatches a fullpointerdown→mousedown→pointerup→mouseup→clicksequence forSimulatedClickCreationScope::kFromAccessibility, andsimulated_event_util.cchardcodes the pointer type:Coordinates are set to the element's border-box center (not
0,0), andisTrustedistrue. So on Chrome, AT activation is indistinguishable from a physical mouse click bypointerType,pointerId,isTrusted, coordinates, ordetail.Incidentally this also identifies the sequence described in #174046 as the Blink path, not Safari.
WebKit / Safari — unaffected.
AccessibilityObject::press()goes through in-enginedispatchSimulatedClick, which emitsmousedown/mouseup/clickand no pointer events at all; the loneclickPointerEvent is spec-compliant withpointerType = ""andpointerId = -1._maybeStartDebouncingis never reached.What that means in practice
On Chrome, a screen-reader activation now takes the new early return, so the subsequent
clickis deduped and noSemanticsAction.tapis sent. Activation instead becomes a real down/up tap at the widget's centroid:GestureDetector,InkWell, most buttons) — still activates, with better fidelity than before.Semantics(onTap:)node with no recognizer under the centroid — silently stops responding to screen-reader activation.That is the same trade described below, but extended to AT users, which raises the stakes a lot. I don't think I should pick the resolution unilaterally. Options as I see them:
SemanticsAction.tap, and recognizer-lessSemantics(onTap:)is rare.isListening == falsebranch already does this. Preserves AT behavior exactly; gives up the 200ms latency win./cc @yjbanov @mdebbar — you two own this code path; I'd rather have your call than guess.
Trade-off (independent of the above)
Stating plainly, since it is also documented at the call site: mouse input can no longer reach
SemanticsAction.tapby any route. The engine cannot know whether the framework consumed the pointer gesture, so there is no version of this fix that keeps both.Only
pointerType == 'mouse'is exempted.penandtouchlose the same detail for the same reason, but touch is the case the debouncer was written for and both were in scope when the 200ms window was tuned, so widening the exemption felt like a bigger swing than this fix warrants.Tests
_testClickDebouncercases migrated fromcontext.primaryDown()/primaryUp()to localtouchDown()/touchUp()helpers, since mouse no longer debounces.pointerdown/pointerupon anflt-tappablenode reach the framework as pointer packets, and the following DOMclickdoes not additionally fire aSemanticsAction.tap.Possibly also fixed
From the code path these look like the same root cause, but I have not reproduced them:
secondarywidget inRadioListTilebecomes noninteractable via mouse in ^3.19.0 #146063 (secondarywidget inRadioListTilebecomes noninteractable via mouse)