Skip to content

Synchronize dispatch_actions in WebDriver#36932

Merged
mrobinson merged 10 commits intoservo:mainfrom
longvatrong111:webdriver-sync
May 21, 2025
Merged

Synchronize dispatch_actions in WebDriver#36932
mrobinson merged 10 commits intoservo:mainfrom
longvatrong111:webdriver-sync

Conversation

@longvatrong111
Copy link
Copy Markdown
Contributor

@longvatrong111 longvatrong111 commented May 9, 2025

Implement missing synchronization in dispatch_actions of WebDriver.
https://w3c.github.io/webdriver/#dispatching-actions

The user agent event loop has spun enough times to process the DOM events generated by the last invocation of the >dispatch tick actions steps.

  • Add a way for ScriptThread to notify WebDriver about the completion of input commands.
  • Add a webdriver_id field for InputEvent. ScriptThread uses it to distinguish WebDriver events and sends notification.

Tests:
./mach test-wpt --product servodriver -r tests\wpt\tests\webdriver\tests\classic\element_click\events.py pass if hit_testing pass. Check issue

cc: @xiaochengh

@longvatrong111 longvatrong111 changed the title Webdriver sync Synchronize dispatch_actions in WebDriver May 9, 2025
Copy link
Copy Markdown
Member

@yezhizhen yezhizhen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An events route exists between WebDriver <-> Constellation <-> ScriptThread. Should we use it to receive response from ScriptThread? It means merge WebdriverData.sync_channel and WebdriverData.load_channel. Other option is let ScriptThread sends reply directly to WebDriver, similar to WebDriverScriptCommand

I prefer the other option. Right now sync_channel in Constellation does nothing except forwarding WebDriverInputComplete from ScriptThread back to WebDriver Server.

Mostly LGTM, except that perform_scroll also requires similar sync like the rest. See the comment below :)

@longvatrong111
Copy link
Copy Markdown
Contributor Author

An events route exists between WebDriver <-> Constellation <-> ScriptThread. Should we use it to receive response from ScriptThread? It means merge WebdriverData.sync_channel and WebdriverData.load_channel. Other option is let ScriptThread sends reply directly to WebDriver, similar to WebDriverScriptCommand

I prefer the other option. Right now sync_channel in Constellation does nothing except forwarding WebDriverInputComplete from ScriptThread back to WebDriver Server.

Mostly LGTM, except that perform_scroll also requires similar sync like the rest. See the comment below :)

Noted. We already have load_channel, need to check more if we can use it.

@longvatrong111 longvatrong111 marked this pull request as ready for review May 13, 2025 21:46
@xiaochengh
Copy link
Copy Markdown
Contributor

Some points to discuss:

  • An events route exists between WebDriver <-> Constellation <-> ScriptThread. Should we use it to receive response from ScriptThread? It means merge WebdriverData.sync_channel and WebdriverData.load_channel. Other option is let ScriptThread sends reply directly to WebDriver, similar to WebDriverScriptCommand

I also prefer the latter. If there's an existing design that works, we should reuse it so that it's easier for future developers.

  • In current implementation, WebDriver creates a temporary pair of Sender and Receiver and pass the Sender with each command to constellation or ScriptThread. Should we make permanent channels?

A temporary pair seems fine (I'm assuming it's the current mechanism in script commands). WebDriver is not performance-sensitive code, so it's better to follow existing design.

@longvatrong111
Copy link
Copy Markdown
Contributor Author

longvatrong111 commented May 14, 2025

Some points to discuss:

  • An events route exists between WebDriver <-> Constellation <-> ScriptThread. Should we use it to receive response from ScriptThread? It means merge WebdriverData.sync_channel and WebdriverData.load_channel. Other option is let ScriptThread sends reply directly to WebDriver, similar to WebDriverScriptCommand

I also prefer the latter. If there's an existing design that works, we should reuse it so that it's easier for future developers.

  • In current implementation, WebDriver creates a temporary pair of Sender and Receiver and pass the Sender with each command to constellation or ScriptThread. Should we make permanent channels?

A temporary pair seems fine (I'm assuming it's the current mechanism in script commands). WebDriver is not performance-sensitive code, so it's better to follow existing design.

ScriptThread handles script command immediately.
For input command, because ScriptThread stores input events to handle later, if we do the same, ScriptThread needs to store ipc channels for each command in correct order.
Are you ok with that way?

In addition, there is an existing route between WebDriver and ScriptThread, WebDriver are using both options.

@longvatrong111 longvatrong111 force-pushed the webdriver-sync branch 3 times, most recently from c775965 to 15684dd Compare May 14, 2025 08:30
@yezhizhen
Copy link
Copy Markdown
Member

Tests: ./mach test-wpt --product servodriver -r tests\wpt\tests\webdriver\tests\classic\element_click\events.py pass if hit_testing pass.

Maybe you can add this link #36676 (comment) in the PR message so that people coming by can better understand the context of hit_testing.

Copy link
Copy Markdown
Contributor

@xiaochengh xiaochengh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general. Just some coding style comments without major concerns.

Copy link
Copy Markdown
Contributor

@xiaochengh xiaochengh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Copy Markdown
Member

@mrobinson mrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a couple notes here. I have not reviewed the webdriver portions as I'm probably not the best person for that and have just focused on the embedding layer.

@longvatrong111
Copy link
Copy Markdown
Contributor Author

Thanks! Just a couple notes here. I have not reviewed the webdriver portions as I'm probably not the best person for that and have just focused on the embedding layer.

I've updated. Thank you!

@longvatrong111
Copy link
Copy Markdown
Contributor Author

...as lot of this will need to change when we move our WebDriver implementation to servoshell.

Sorry if I'm out of sync. I thought WebDriver should not depend on servoshell?

It is under discussion in #36714.
You can check the last comment .

@mrobinson
Copy link
Copy Markdown
Member

If we can enable WebDriver conformance check on CI, it is a huge convenience for later WebDriver development.

I definitely agree with this. I think maybe we need a more generic mechanism for this in the WebView API. Currently you send an event to the WebView via WebView::notify_input_event and then unprocessed key events are reported via WebViewDelegate::notify_keyboard_event. Likely we need some sort of WebViewDelegate::notify_processed_input_event which reports whether it was handled or not. In the case it wasn't handled, the embedder is responsible for sending the event back to the toolkit. If WebView::notify_input_event is responsible for setting an id on the internal input event and returning it, WebViewDelegate::notify_processed_input_event can also include that id. This could be used by WebDriver to accomplish what you are doing here.

@mrobinson
Copy link
Copy Markdown
Member

Sorry if I'm out of sync. I thought WebDriver should not depend on servoshell?

Ultimately, WebDriver should be testing the embedding API, so it will need some way of controlling servoshell in order to create and add WebViews to the interface. This could be done either by making the webdriver crate part of servoshell or asking for callbacks which servoshell provides (to avoid a dependency). Right now having webdriver sitting in the middle of the Servo stack is causing a lot of issues.

@mrobinson
Copy link
Copy Markdown
Member

Oh, I forgot to mention that an added benefit of having WebDriver use the embedding API is that it ensures that the embedding API is rich enough.

Copy link
Copy Markdown
Member

@mrobinson mrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I've pushed a small commit that fixes a couple nits so minor I didn't want to bother you with them. Thank you so much for the patience here and apologies for the long review process.

@mrobinson mrobinson enabled auto-merge May 21, 2025 09:27
Signed-off-by: Martin Robinson <[email protected]>
@longvatrong111
Copy link
Copy Markdown
Contributor Author

Thank you! I've pushed a small commit that fixes a couple nits so minor I didn't want to bother you with them. Thank you so much for the patience here and apologies for the long review process.

It is totally ok. Thanks for your fixing.

@mrobinson mrobinson added this pull request to the merge queue May 21, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 21, 2025
@mrobinson
Copy link
Copy Markdown
Member

Looks like there are a couple build issues on OpenHarmony which must use the new constructor for embedder input events.

@longvatrong111 longvatrong111 requested a review from jschwe as a code owner May 21, 2025 10:31
@mrobinson mrobinson enabled auto-merge May 21, 2025 10:32
@mrobinson mrobinson added this pull request to the merge queue May 21, 2025
Merged via the queue into servo:main with commit f52fa9b May 21, 2025
21 checks passed
@longvatrong111 longvatrong111 deleted the webdriver-sync branch May 21, 2025 13:31
yezhizhen added a commit to yezhizhen/servo that referenced this pull request May 22, 2025
github-merge-queue bot pushed a commit that referenced this pull request May 23, 2025
…_list` (#37081)

Re-remove deleted logic from `WebDriverSession::input_cancel_list` logic
in #37010 which was re-adedd during merge resolution in #36932.
Otherwise, duplicate items would be registered in Release Actions.

Signed-off-by: Euclid Ye <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jun 11, 2025
Implement action synchronization for wheel event. Previously only done
for pointer here #36932.

Testing:
`tests/wpt/meta/webdriver/tests/classic/perform_actions/wheel.py`

---------

Signed-off-by: PotatoCP <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jun 18, 2025
Implement action synchronization for key event. Previously only done for
pointer #36932 and wheel
#37260.

---------

Signed-off-by: PotatoCP <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants