Support Selenium 4.0 Grid CDP for Devtools Service by dylanlive · Pull Request #6508 · webdriverio/webdriverio

dylanlive · 2021-02-25T00:29:19Z

Proposed changes

Addresses #6470. This PR allows using the devtools service with Selenium 4.0 Grid (beta-1).

When creating a new session, Selenium 4.0 Beta 1 will respond with an endpoint that can be used to make direct websocket requests with (see below for example response).

Puppeteer takes that as browserWSEndpoint as described here.

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update

Checklist

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)
I have added proper type definitions for new commands (if appropriate)

Further comments

I have read discussions that potentially in a future Se4 Beta, the structure of "se:options" may change. However, I figured we can at least get the ball rolling now and make tweaks as needed.
The conditional precedence is very important in this PR. A response can have both se:options and the debuggerAddress that gets tacked on by Chromedriver.

Full response example here:

{
    "status": 0,
    "sessionId": "c8adfc6384648299d726d9c63d4e7a15",
    "value": {
        "acceptInsecureCerts": false,
        "browserName": "chrome",
        "browserVersion": "88.0.4324.182",
        "chrome": {
            "chromedriverVersion": "87.0.4280.20 (c99e81631faa0b2a448e658c0dbd8311fb04ddbd-refs/branch-heads/4280@{#355})",
            "userDataDir": "/var/folders/kx/c8_k_9q17qngwmlbc282nhc0bnj1_9/T/.com.google.Chrome.WKVL7Y"
        },
        "goog:chromeOptions": {
            "debuggerAddress": "localhost:52810"
        },
        "networkConnectionEnabled": false,
        "pageLoadStrategy": "normal",
        "platformName": "mac os x",
        "proxy": {},
        "se:options": {
            "cdp": "http://my.grid:5555/session/c8adfc6384648299d726d9c63d4e7a15/se/cdp"
        },
        "setWindowRect": true,
        "strictFileInteractability": false,
        "timeouts": {
            "implicit": 0,
            "pageLoad": 300000,
            "script": 30000
        },
        "unhandledPromptBehavior": "dismiss and notify",
        "webauthn:virtualAuthenticators": true
    }
}

(just for note) I'm hoping [Selenium 4] [Grid] Se:Options CDP Endpoint Points to Node SeleniumHQ/selenium#9202 is addressed. Personally I prefer the Selenium Hub to proxy all cdp requests. Beta 1 currently returns a direct endpoint to the Node. However, I feel it would be irresponsible in this PR to hardcode the hub (via something like ws://${this.config.hostname}:${this.config.port}/session/${this.sessionId}/se/cdp, which works) instead of utilizing what is returned by Selenium.

Reviewers: @webdriverio/project-committers

dylanlive · 2021-02-25T00:44:16Z

Not seeing where to sign the CLA in any of the contributor docs.

L0tso · 2021-02-25T08:32:09Z

Not seeing where to sign the CLA in any of the contributor docs.

https://cla.js.foundation/webdriverio/webdriverio?pullRequest=6508

dylanlive · 2021-02-25T18:29:58Z

+    const cdpEndpoint = caps['se:options']?.cdp
+    if (cdpEndpoint) {
+        this.puppeteer = await puppeteer.connect({
+            browserWSEndpoint: cdpEndpoint


If we make some hardcoded assumptions here, this could work with the docker images as described in #6470 (comment)

Something like:
ws://${this.config.hostname}:${this.config.port}/session/${this.sessionId}/se/cdp

Definitely not as slick though and more likely to break in a future update. Would be a temp bandaid until SeleniumHQ/selenium#9202 is addressed.

christian-bromann

One comment but otherwise this looks good to me. Do you mind signing the CLA?

christian-bromann · 2021-03-01T08:54:35Z

+     * Selenium 4.0 Specific
+     */
+    'se:options'?: {
+        cdp?: string;


Do we know more Selenium specific options we can add within this PR?

Not to my knowledge

dylanlive · 2021-03-01T16:45:34Z

Will get the CLA signed as soon as I'm able to

christian-bromann · 2021-03-02T10:04:12Z

@dylanlive awesome! It would be also great if you could update your branch as I tweaked the pipline a bit. Thanks!

christian-bromann · 2021-03-03T08:03:28Z

@dylanlive any chance to update the branch and sign the CLA? Would love to get this into the next v7.1 release.

Selenium 4 (beta-1) Grid supports CDP Websocket Requests, and returns the endpoint in the returned capabilities under {"se:options": { "cdp": "the_url" }}. This commit will utilize that endpoint (if returned in the capabilities) as the Puppeteer Websocket Endpoint.

dylanlive · 2021-03-05T18:39:26Z

@christian-bromann CLA signed, thanks for your patience. Merged in the latest changes and applied the defaultViewport from #6487

Did a few quick tests as well with a Se4 Grid (confirmed working) and a Se3 Grid (confirmed not working, as expected).

As a suggestion, if you could add a link to the CLA in https://github.com/webdriverio/webdriverio/blob/main/CONTRIBUTING.md, I think that could have sped up the process in my case.

christian-bromann · 2021-03-06T12:05:05Z

As a suggestion, if you could add a link to the CLA in https://github.com/webdriverio/webdriverio/blob/main/CONTRIBUTING.md, I think that could have sped up the process in my case.

Good idea, will make that change.

Thanks for the contribution!

christian-bromann

👍

tripuzv · 2023-06-21T16:41:23Z

I have a problem with debuggerAddress, cause I am using https instead of http. How to fix it?
When i use only host it returns to me HTTP Bad Request,
But if i use whole address with https it trying to connect next address http://https/json/version:

christian-bromann · 2023-06-22T06:55:20Z

Why are you using https for a local Selenium server?

dylanlive mentioned this pull request Feb 25, 2021

Fix shrinked viewport for devtools package #6487

Merged

8 tasks

dylanlive commented Feb 25, 2021

View reviewed changes

christian-bromann reviewed Mar 1, 2021

View reviewed changes

christian-bromann mentioned this pull request Mar 1, 2021

[wdio-devtools-service] Support CDP on Selenium 4 Grid #6470

Closed

christian-bromann linked an issue Mar 1, 2021 that may be closed by this pull request

[wdio-devtools-service] Support CDP on Selenium 4 Grid #6470

Closed

christian-bromann mentioned this pull request Mar 5, 2021

Use a remote Chrome browser with DevTools Protocol #6550

Closed

dylanlive added 3 commits March 5, 2021 10:16

syntax improvements for capabilities

88029eb

Update with default viewport

4c72aba

dylanlive force-pushed the se_4_cdp branch from a566826 to 4c72aba Compare March 5, 2021 18:35

christian-bromann added the PR: New Feature 🚀 PRs that contain new features label Mar 6, 2021

christian-bromann approved these changes Mar 6, 2021

View reviewed changes

christian-bromann merged commit 157ace4 into webdriverio:main Mar 6, 2021

esaari mentioned this pull request Jun 28, 2022

[📖 Docs]: Add information that mock commands are only working for local tests #8467

Closed

2 tasks

b49tan mentioned this pull request Jul 26, 2022

Setting custom headers does not work on CICD/jenkins #8591

Closed

BorisOsipov mentioned this pull request Nov 8, 2022

[devtools] [selenium grid] Should the dev-tools service work with Selenium Grid? #4795

Closed

Uh oh!

Conversation

dylanlive commented Feb 25, 2021

Proposed changes

Types of changes

Checklist

Further comments

Reviewers: @webdriverio/project-committers

Uh oh!

dylanlive commented Feb 25, 2021

Uh oh!

L0tso commented Feb 25, 2021

Uh oh!

dylanlive Feb 25, 2021

Choose a reason for hiding this comment

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

christian-bromann Mar 1, 2021

Choose a reason for hiding this comment

Uh oh!

dylanlive Mar 1, 2021

Choose a reason for hiding this comment

Uh oh!

dylanlive commented Mar 1, 2021

Uh oh!

christian-bromann commented Mar 2, 2021

Uh oh!

christian-bromann commented Mar 3, 2021

Uh oh!

dylanlive commented Mar 5, 2021

Uh oh!

christian-bromann commented Mar 6, 2021

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

tripuzv commented Jun 21, 2023

Uh oh!

christian-bromann commented Jun 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants