You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit aa3e869
Browse filesBrowse the repository at this point in the historyBrowse files
Dmitrii Zolotov
committed
Add the mouse_capture plugin: relative pointer deltas on macOS
Flutter exposes no pointer lock on any desktop platform, which makes a
first-person camera impossible out of the box: the cursor reaches the edge of
the window and the view stops turning. flutter/flutter#134652 is open.
The mechanism is the one AppKit intends. Turning off the association between
the physical mouse and the on-screen cursor freezes the cursor while mouseMoved
events keep arriving with their deltas intact; hide the cursor, read deltaX and
deltaY from a local NSEvent monitor, and that is all of it. Warping the cursor
back to a fixed point every frame also works but fights the window server and
shows as jitter, so it stays a fallback.
helgoboss/pointer_lock (MIT, unpublished on pub.dev) already does this across
four platforms and is where the technique came from. This is a separate plugin
because three of its gaps matter to a game: no observer for focus loss, so
Cmd+Tab strands the cursor system-wide; one channel message per mouse event at
a 1000 Hz polling rate; and no isSupported, which a build targeting mobile
needs.
The delta is drained with takeDelta rather than pushed on a stream. That is the
shape a fixed-timestep simulation wants — it asks how far the mouse moved since
the last step, once per step — and it is synchronous, because awaiting inside
the step would mean the step no longer sees a consistent snapshot of its inputs.
Two traps, both now covered. A plugin outlives the Dart isolate, so a hot
restart while captured leaves the cursor hidden with nothing left to ask for it
back; construction issues a reset, which is only safe because the native side
keeps its own flag — hide and unhide are counted, and an unbalanced unhide
quietly breaks the next hide. And focus loss must release rather than pause,
or a hidden cursor ends up over another application's window.
A package rather than app code: it knows nothing about the engine or the game
and can be lifted out whole. The example app exists to isolate it — capture
fails in ways that look exactly like a broken camera, and telling those apart
inside a 3D scene means debugging two things at once.
17 tests on the Dart accumulator and state machine, which is where the
easy-to-miss failures live. The native side is verified by running the example.
Copy file name to clipboardExpand all lines: README.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -233,6 +233,8 @@ either does not start or silently renders nothing. Worth keeping to hand.
233
233
| Shadows look right but toggling them changes nothing | Check the setting actually reaches `RenderSettings`. A control wired to the panel but not to the renderer looks completely convincing, and the way to find out is to capture the same frame with the feature on and off and diff the two — a zero difference is the whole answer |
234
234
| A draw is submitted, the counter goes up, nothing appears | There is **no non-indexed draw**. `draw()` with only a vertex buffer bound succeeds and renders nothing. Bind an index buffer even when the indices are the identity `0, 1, 2, …` sequence — the debug line overlay keeps one in a device buffer that only grows |
235
235
|`PathAccessException … Operation not permitted` when writing a file | macOS Flutter apps are sandboxed. Anything outside `~/Library/Containers/<bundle id>/Data` is refused, so frame captures resolve relative paths against the app's own temp directory |
236
+
| The view stops turning when the cursor reaches the edge of the window | Flutter exposes no pointer lock on any desktop platform. `packages/mouse_capture` supplies it on macOS by turning off the association between the physical mouse and the on-screen cursor, which leaves `mouseMoved` events arriving with their deltas while the cursor stays put |
237
+
| No cursor anywhere after a hot restart | A plugin holding the pointer outlives the Dart isolate, because the engine's registrar owns it. The Dart side comes back remembering nothing, so nothing asks for the cursor back. `MouseCapture` issues a reset on construction for exactly this |
Copy file name to clipboardExpand all lines: RESEARCH.md
+48Lines changed: 48 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -618,6 +618,52 @@ What remains unverified: a benchmark of draw calls and GC pauses at 10k transfor
618
618
619
619
---
620
620
621
+
## 6. Pointer capture, which the framework does not provide
622
+
623
+
Recorded here because it is a gap in Flutter rather than in `flutter_gpu`, and
624
+
because a first-person camera is impossible without closing it: the cursor
625
+
reaches the edge of the window and the view stops turning. The framework exposes
626
+
no pointer lock on **any** desktop platform, and the request for one
627
+
([flutter/flutter#134652](https://github.com/flutter/flutter/issues/134652)) is
628
+
open.
629
+
630
+
**What works on macOS.** Turning off the association between the physical mouse
631
+
and the on-screen cursor —
632
+
`CGAssociateMouseAndMouseCursorPosition(0)` — freezes the cursor while
633
+
`mouseMoved` events keep arriving with their deltas intact. Hide the cursor with
634
+
`NSCursor.hide()`, read `event.deltaX/deltaY` from a local `NSEvent` monitor, and
635
+
that is the whole mechanism. Warping the cursor back to a fixed point every frame
636
+
also works, but it fights the window server and shows as jitter, so it is a
637
+
fallback and not the design.
638
+
639
+
This was not guesswork: [helgoboss/pointer_lock](https://github.com/helgoboss/pointer_lock)
640
+
(MIT, unpublished on pub.dev) already does it this way across four platforms, and
641
+
is where the technique came from. Our own `packages/mouse_capture` exists because
642
+
three of its gaps matter to a game — no observer for focus loss, so Cmd+Tab
643
+
strands the cursor system-wide; one platform-channel message per mouse event at a
644
+
1000 Hz polling rate; and no `isSupported`, which a build targeting mobile needs.
645
+
646
+
**Two traps worth naming.**
647
+
648
+
*The cursor outlives the Dart isolate.* A plugin is owned by the engine's
649
+
registrar, so a hot restart while the pointer is captured leaves the native side
650
+
still holding it with nothing left that remembers to ask for it back — no cursor,
651
+
and no way to recover except quitting. The fix is a `reset` issued on
652
+
construction, and it only works because the native side keeps its own flag: hide
653
+
and unhide are counted, so unhiding without having hidden would leave the count
654
+
negative and quietly break the next hide.
655
+
656
+
*Focus loss must release, not pause.* Anything else leaves a hidden cursor over
657
+
another application's window. Both `NSApplication.didResignActive` and
658
+
`NSWindow.didResignKey` are needed — the first fires on Cmd+Tab, the second when
659
+
another window of the same app takes focus.
660
+
661
+
**Not yet measured**: whether one channel message per mouse event is visible in a
662
+
frame profile. If it is, the fix is to accumulate natively and flush once per
-[[Impeller] Support compute passes/shaders — flutter/flutter#109346](https://github.com/flutter/flutter/issues/109346)
683
+
-[Flutter should support capturing the mouse pointer — flutter/flutter#134652](https://github.com/flutter/flutter/issues/134652)
# The .vscode folder contains launch configuration and tasks you configure in
22
+
# VS Code which you may wish to be included in version control, so this line
23
+
# is commented out by default.
24
+
#.vscode/
25
+
26
+
# Flutter/Dart/Pub related
27
+
# Libraries should not include pubspec.lock, per https://dart.dev/guides/libraries/private-files#pubspeclock.
0 commit comments