Repository navigation
Description
When embedding a FlutterView onto a custom element, e.g. in multi-view mode, the app may have two sources of scrolling:
- Flutter scrolls some content inside itself.
- The host page scrolls its content, and one piece of content may be a
FlutterView.
Case 1 seems to work well. However, case 2 does not work when the mouse cursor or finger (i.e. pointer) is on top of the FlutterView. When this happens the web engine calls preventDefault, which prevents browser scrolling. Somehow the engine needs to know when not to call preventDefault, but that information lives in the framework.
Demo: in the sample below, see that when the pointer is on top of the blue area the page is able to scroll, but if the pointer moved to the yellow area the scrolling no longer happens.
Screen.Recording.2023-11-29.at.12.54.43.PM.mov
A couple of potential fixes:
- Change PointerDataPacketCallback to return some value to the engine that provides information about whether the pointer should be forwarded to the platform for handling.
- As part of rendering the frame, perhaps in the
Scenegraph, annotate regions with information about how certain pointer events should be handled.
My gut feeling tells me that the first option is better. For one, it is simpler and less invasive. But also, the logic that determines what to do with a pointer may depend on many things - kind, device, location, pressure - and the second option would have to pre-declare all this information even if a tiny subset of it is applicable.
This may require a collaboration between the framework and the web teams, but I'm assigning it to the framework team first to get guidance on what option would be the most idiomatic and work well across platforms.
Activity
goderbauer commentedon Nov 29, 2023
Some thoughts about this:
Based on a single pointer event (e.g. a pointer down) we cannot decide whether the framework will "handle" the event. For example, when we see a pointer down event delivered via the PointerDataPacketCallback this could be part of a tap gesture (which we want to handle in the framework) or it could end up being a scroll (which we may want to handle in the browser). We will only know for sure after we have seen a sequence of follow-up events and only at that point can we tell you whether the framework handled them or not (basically, once we know if any gesture handler claimed victory in the gesture arena). This information wouldn't be available synchronously when pointer events are delivered to the framework.
I think this has the same problem, but in reverse: How are you on the engine-side going to predict that a pointer down event is going to be a scroll gesture (and needs to be handled directly without telling the framework) vs. its going to be a tap gesture that the framework needs to know about to activate a button?
Somehow, the gesture arena of the framework needs to work together with the gesture recognition system of the browser. Not sure how we can archive this, though, and if browsers have any APIs for this?
/cc @mdebbar
flutter-triage-bot commentedon Dec 1, 2023
The
triaged-weblabel is irrelevant if there is noteam-weblabel orfyi-weblabel.goderbauer commentedon Jan 4, 2024
Earlier @yjbanov and I had a conversation about this and came up with two potential ideas:
Idea: Competition
We send a specially crafted member representing the browser into the gesture arena for every pointer. It can only win if it is the last remaining member and no other member wins. It remembers all the pointer events dispatched to it and the platform prevents defaults for all of them. When this special member "wins" the arena, we play the events back to the platform and let it handle it since nobody on the framework side wants them.
This will require some API on the platforms to inject events back into the platform. Such API may not exist.
Idea: Cooperation
For scroll wheel events we could check whether the
PointerSignalResolverhas any registered event handlers whenPointerSignalResolver.resolveis called. If it has a registered handler for the event, then the event is handled by the framework and the platform is told synchronously to prevent defaults. If it has no registered handler, the platform needs to handle the event. This should basically resolve the problem altogether for scroll wheels because Scrollables only register a handler with the PointerSignalResolver if they can actually scroll in response to the event.For scroll by touch we could extend
GestureArenaMemberwith an option to prevent defaults when a particular member enters the gesture arena on a point down event. By default, all GestureDetectors would allow defaults on theGestureArenaMemberthat they send to the arena - except for the member added by a vertically scrolling Scrollable. Pointer events are now dispatched as usual, but when at the end of dispatching a pointer down event the arena is closed we check if any members disallow defaults. If that is the case, the platform is told to prevent defaults, otherwise it isn't. This isn't expected to solve the issue in all cases, though: If a scrollable is all the way at the top it can still scroll down and would therefore send a default-preventing member into the arena on pointer down to prevent defaults even when the pointer down event would be the start of a scroll up gesture that should be handled by the platform, but wouldn't be in this scenario. An argument could be made here that UIs with nested scrolling are kinda bad and should be avoided. Either you want the FlutterView to be scrollable or the outer page, but not both at the same time. Therefore, this could be a "good enough" solution that fulfills most use cases.Both of these rely on the fact that upon receiving either a signal or a pointer down event from the platform we can synchronously message back to the platform whether defaults need to be prevented. In the framework, event handling is pretty much synchronous - with two known exceptions: locked events and resampling.
When events are locked incoming pointer events are queued and processed after events are unlocked again. Currently, we only lock events while producing a warmup frame and during hot reload. Hot reload isn't available yet on the web, so we may not have to worry too much about it. And even if it were, it would probably be fine if events arriving during this dev operation behave a little strange - after all, the app is reassembling itself. The same can be said about the warm-up frame right at app startup (and after hot reload).
If resampling is enabled (it is off by default) we may be out of luck since the whole point of it is to process event asynchronously. PointerSignalEvents appear to not be affected by resampling, so those could still work. It will need some additional investigation to see if PointerDown events are also dispatched asynchronously as part of resampling.
There may be further asynchronicity in event dispatch on the engine side. At least when semantics are enabled, the web engine seems to operate in async mode for event dispatch (see here). Further investigation is needed to figure out if we could make that work with this approach.
There are also the other non-web platforms to consider (supporting multi-view in an add-to-app fashion will surface the same problems there). They will likely involve a thread hop for event dispatch and further investigation will be needed to see if defaults can be prevented asynchronously for incoming pointer events.
goderbauer commentedon Jan 4, 2024
Idea "competition" is kinda of how PlatformViews work today on at least iOS and (in the future) MacOS. It could likely also be used for view embedding on those platforms. (cc @cbracken)
@yjbanov How does the web do gesture disambiguation with platform views there to decide whether the platform view or the surrounding flutter app should handle it?
mdebbar commentedon Jan 16, 2024
@goderbauer thanks for the write up!
AFAIK, this is not possible on the web. We can create an event and dispatch it to the DOM, but the browser won't handle it.
Note: after playing a bit with this in a jsfiddle, I found out the event that needs to be
preventDefaulted istouchmovewith{passive: false}. Preventing default onpointermove, for example, won't work.It is my understanding that on the web, platform views always swallow pointer events. There are a few reasons:
The solution we have right now is to let the app developer decide. If they want to let the platform view handle the events, then they don't have to do anything (this is the default behavior). But if they want to handle the events on the Flutter side, they can use
pointer_interceptor.yjbanov commentedon Jan 17, 2024
A-ha! That's why it didn't work in my experiments. Thanks for digging this up!
14 remaining items
ditman commentedon May 21, 2024
@Garoth full-screening the Flutter view is a cool feature!
[web] Adds allowPlatformDefault for wheel signals. (#51566)
[web] Notify engine of handled PointerScrollEvents. (#145500)
ditman commentedon Jun 29, 2024
@Garoth looking at your code, I think this PR may be relevant to you:
[web] Fixes drag scrolling in embedded mode. (#53647)
yjbanov commentedon Jul 17, 2024
Scrolling using the mouse and the touchpad has landed, which was in the MVP of the feature. Touch-based scrolling can be addressed as a follow-up.
github-actions commentedon Aug 1, 2024
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.