webdriver: Add touch support for all platforms#41067
Conversation
|
🔨 Triggering try run (#19956493099) for Linux (WPT) |
|
🔨 Triggering try run (#19956497303) for Linux (WPT), OpenHarmony |
Signed-off-by: Euclid Ye <[email protected]>
b71e42a to
039f834
Compare
|
Could this get an automated test to prevent future regressions with a simple demo page that only has a touch handler (is there no wpt test already covering this)? |
Maybe later after this PR. |
Signed-off-by: Euclid Ye <[email protected]>
|
🔨 Triggering try run (#19956792916) for OpenHarmony |
I don't think there is for "element click", as it is not supported by any vendor yet and is under ongoing discussion in W3C. But there are for "Perform actions". I haven't supported it yet, which would be in a future PR. It is used less frequently but still important. |
|
Test results for linux-wpt from try job (#19956493099): Flaky unexpected result (33)
Stable unexpected results that are known to be intermittent (20)
Stable unexpected results (138)
|
|
|
|
✨ Try run (#19956792916) succeeded. |
|
Oh the impact of this, is much deeper than I thought. I need to fix the "Got a touchmove event for a non-active touch point" later. We can cover a lot more test with this change after it! |
All these tests never worked as intended, as we've been ignoring the "touch" pointer type, always dispatching mouse. I would like to update the test results for now, and improve this very soon later. |
| .or_insert(InputSourceState::Pointer(PointerInputState::new( | ||
| PointerType::Mouse, | ||
| parameters.pointer_type, |
There was a problem hiding this comment.
We are no longer ignoring requested pointer type.
This allows us to dispatch touch actions correctly for Perform Actions.
Signed-off-by: Euclid Ye <[email protected]>
|
🔨 Triggering try run (#19957800954) for Linux (WPT) |
|
Test results for linux-wpt from try job (#19957800954): Flaky unexpected result (27)
Stable unexpected results that are known to be intermittent (20)
|
|
✨ Try run (#19957800954) succeeded. |
Actually I have investigated it a little bit. I think we assume that servo/components/compositing/touch.rs Lines 333 to 353 in 78fd696 and this will fails servo/components/compositing/touch.rs Lines 429 to 439 in 78fd696 |
Signed-off-by: Euclid Ye <[email protected]>
jschwe
left a comment
There was a problem hiding this comment.
LGTM, hope to see the follow-ups soon 👍
| /// There is currently no spec for touchscreen webdriver support. | ||
| /// There is an ongoing discussion in W3C: | ||
| /// <https://github.com/w3c/webdriver/issues/1925> | ||
| #[cfg(any(target_env = "ohos", target_os = "android"))] |
There was a problem hiding this comment.
I would prefer if we did a runtime check for device type, since ohos / android devices can be run in desktop mode.
I don't think we need ot change this now, would just prefer if we didn't add to many cfgs like this when it shouldn't be a compiletime option.
There was a problem hiding this comment.
Thanks! Will do when I got time.
Another way is to process "user_agent" of webdriver session to determine at runtime.
There was a problem hiding this comment.
TODO later: When we have touch simulation for Desktop, "Element click" should also dispatch pointer touch.
We always have a plan to do this in runtime.
…bedder `MouseMove` (#41171) Back in #41067, we added webdriver touch support for all platforms. All touch-related testdriver tests which faked to work has been CRASH instead, as we assume that touch down event always happens before touch move and touch up. But the assumption above is not wrong, as I observed embedder events dispatched from OHOS/Android when interacting as human. It is just that the tests are flawed: almost all of them have pointermove first even for touch, and spec didn't consider touch/pen support well enough: #41042. Luckily, our embedder `MouseMove` event would also update the webdriver pointer status, which effectively allow touch down/touch up to happen at right point. It **doesn't matter** whether the website has mouse listener or not. Testing: Tested manually on a touch-event-only complicated website. Also, tests no longer crash, also with expectations different from before #41067, e.g. [this](https://github.com/servo/servo/pull/41067/files#diff-db149f4025dab43bd5a3c8a412ea2ac9e521f4745c0f982925b4d15b236ff955) had been failing but now passes. --------- Signed-off-by: Euclid Ye <[email protected]>
Some mobile websites only have touch event listeners, but not any for mouse events. We add native touch support for "element click", for Android/OHOS. We are finally utilizing
subtypeand can remove thedead_codemacro for it: we dispatch action based on the subtype given, which has impact for Perform Actions on all platforms.Testing:
Fixes: The step 4 of #41042. We still need to fix some bugs of spec, and wait for resolution.
TODO later: When we have touch simulation for Desktop, "Element click" should also dispatch pointer touch.