Repository navigation
Commit 9e619e1
authored
Add real WebDriver pointer actions to package:integration_test on the web (flutter#195207)
Adds a first-class way for Flutter web integration tests to drive the
*real* browser mouse through WebDriver: clicks, hovers, drags, and
arbitrary pointer action sequences with any mouse button.
`WidgetTester.tap`, `WidgetTester.drag` and `binding.handlePointerEvent`
inject synthetic events straight into the framework. They skip the
browser's own DOM hit testing and event dispatch, which is what decides
whether a pointer event lands on Flutter content or on a platform view
(`<iframe>`, map, video) underneath it. So features that depend on
browser hit testing, such as pointer handling over platform views,
couldn't be checked end to end. Until now the only workaround was to
misuse `binding.takeScreenshot` as an RPC and drive the mouse from
inside the host's `onScreenshot` callback.
### New API
On `IntegrationTestWidgetsFlutterBinding`:
- `Future<void> webDriverPointerActions(List<WebDriverPointerAction>
actions, {FlutterView? view})` performs `actions` in order as one real
browser pointer sequence and completes once the browser has dispatched
every event in it.
- `Future<void> webDriverClick(Offset position, {FlutterView? view})` is
a convenience wrapper that moves to `position` and presses and releases
the primary button.
In `package:integration_test/common.dart` (which `integration_test.dart`
does not re-export, so tests import it directly):
- `sealed class WebDriverPointerAction`, created with
`WebDriverPointerAction.move(x, y, {Duration duration})`,
`.down({WebDriverMouseButton button})`, `.up({WebDriverMouseButton
button})` and `.pause(Duration duration)`. The concrete classes are
`WebDriverPointerMove`, `WebDriverPointerDown`, `WebDriverPointerUp` and
`WebDriverPointerPause`, with `toJson`/`fromJson`, value equality and
readable `toString`.
- `enum WebDriverMouseButton { primary, middle, secondary }`, whose
`w3cButton` is the W3C button index (`0`, `1`, `2`), not a Flutter
`kPrimaryButton`-style bit flag.
- `WebDriverCommandType.pointerActions` and
`WebDriverCommand.pointerActions(actions)` in the web command protocol.
- `CallbackManager.pointerActions(List<WebDriverPointerAction>
actions)`, which completes with an `UnsupportedError` by default.
For example, a drag split across two calls so the test can assert on the
intermediate state:
```dart
final Offset start = tester.getCenter(find.byKey(dragHandleKey));
await binding.webDriverPointerActions(<WebDriverPointerAction>[
WebDriverPointerAction.move(start.dx, start.dy),
const WebDriverPointerAction.down(),
WebDriverPointerAction.move(start.dx + 100, start.dy, duration: const Duration(milliseconds: 300)),
]);
// ...assert mid-drag...
await binding.webDriverPointerActions(<WebDriverPointerAction>[
const WebDriverPointerAction.up(),
]);
```
### How it works
- Move positions are logical positions in the target view (for example
from `tester.getCenter`). `webDriverPointerActions` converts them to
viewport CSS pixels by adding the view host element's
`getBoundingClientRect()` origin, so full-page, embedded and multi-view
apps all work. The command travels over the same handshake as
`screenshot`.
- `integrationDriver` (in `integration_test_driver_extended.dart`) sends
the whole sequence as a single W3C perform-actions request for one mouse
input source and replies only after WebDriver has performed it. Tests
can assert right after the `await` with no fixed sleeps, and no
`onScreenshot` callback is needed.
- ChromeDriver performs a timed `pointerMove` as a single `mousemove` at
the end. To make the browser see intermediate events during drags and
hovers, the driver splits each timed move into straight-line steps of
about 16 ms, starting from the previous move's position (remembered
across calls within one `integrationDriver` run) and capped at one step
per pixel. Positions are rounded to whole CSS pixels.
- Pressed buttons stay pressed across calls until released with `up`.
- Invalid input fails fast in the browser: an empty sequence, non-finite
move coordinates, or negative durations throw `ArgumentError` before
anything is sent, and calling without a `view` when there is no implicit
view throws `StateError`. Off the web, both methods throw
`UnsupportedError`.
- If the driver isn't a `WebFlutterDriver`, the host fails the
in-browser call and throws a `StateError`. If WebDriver throws (for
example, coordinates out of bounds), the host releases pressed buttons,
fails the in-browser call, and rethrows the original error, so the test
fails at once instead of hanging.
- Neither method changes `binding.shouldPropagateDevicePointerEvents`.
Real browser events reach the test binding as device events, so a test
that asserts on Flutter widget gestures has to opt in with
`binding.shouldPropagateDevicePointerEvents = true`. Platform view DOM
listeners get the native events either way.
### Tests
- New web e2e `webdriver_pointer_actions_integration`, registered in
`run_web_tests.dart` for CanvasKit and `--wasm` in profile mode. It
checks that a real click reaches a same-origin `<iframe>` platform view
(which `tester.tap` can't do), that a click reaches a Flutter button
only once device pointer events are opted in, that position conversion
still works when the view's host element is offset in the page, that a
timed drag produces intermediate pointer moves, that buttons stay
pressed across calls for a split drag, and that moving without a button
hovers.
- New unit tests in
`packages/integration_test/test/webdriver_command_test.dart` cover
action serialization, equality and `fromJson` validation, the
`pointerActions` command, the `CallbackManager` default, input
validation, and the off-web `UnsupportedError`.
- New unit tests in
`packages/integration_test/test/integration_driver_pointer_actions_test.dart`
cover the W3C request `integrationDriver` builds (rounding, timed-move
subdivision across requests, pauses and buttons) and its error handling
for non-web drivers, WebDriver failures and malformed action JSON.
### Compatibility
`CallbackManager.pointerActions` has a default body, so classes that
`extend` `CallbackManager` keep compiling. A class that `implements` it
would need to add `pointerActions`. The binding's `callbackManager`
can't be swapped out, though, so outside code shouldn't have a real
implementation. `common.dart` is not re-exported from
`integration_test.dart`, so no new names enter the scope of existing
tests.
Part of flutter#179806
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant in-code documentation (doc comments with
`///`).
- [x] If this PR introduces a new feature or capability, I created and
linked a website documentation issue or PR in [flutter/website] (or
verified none is needed).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.
**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/CONTRIBUTING.ai.md
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[flutter/website]: https://github.com/flutter/website
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md1 parent 11e802e commit 9e619e1
11 files changed
Lines changed: 2039 additions & 6 deletions
File tree
- dev
- bots/suite_runners
- integration_tests/web_e2e_tests
- lib
- test_driver
- packages/integration_test
- lib
- src
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
166 | 177 | | |
167 | 178 | | |
168 | 179 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
0 commit comments