Skip to content

[🐛 Bug]: Constantly hanging connections and threads created by dev tools, holding connections on jdk11 http client. #11868

@pogrebnyaksa

Description

@pogrebnyaksa

What happened?

Constantly hanging connections and threads created by dev tools, holding connections on jdk11 http client.

How can we reproduce the issue?

public static void main(String[] args) throws InterruptedException {
        ChromeOptions chromeOptions = new ChromeOptions();
        chromeOptions.addArguments("--ignore-certificate-errors",
                "--start-maximized",
                "--no-sandbox",
                "--dns-prefetch-disable",
                "--disable-gpu");

        System.setProperty("webdriver.http.factory", "jdk-http-client");
        var start = Instant.now();
        Duration loopDuration = Duration.ofMinutes(3);
        while (Instant.now().minus(loopDuration.toMillis(), ChronoUnit.MILLIS).isBefore(start)) {
            WebDriver driver = null;
            try {
                driver = RemoteWebDriver.builder()
                        .augmentUsing(new Augmenter())
                        .oneOf(chromeOptions)
                        .address(System.getProperty("GRID_URL"))
                        .build();
                try (DevTools devTools = ((HasDevTools) driver).getDevTools()) {
                    devTools.createSession();
                    devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
                    devTools.send(Network.setExtraHTTPHeaders(new Headers(Map.of("test", Boolean.TRUE.toString()))));
                    devTools.addListener(Network.requestWillBeSent(), event -> {

                    });
                    driver.get("https://cnn.com");
                }
            } finally {
                if (driver != null) {
                    driver.close();
                }
            }
            TimeUnit.SECONDS.sleep(3);
        }
        while (true) {
            TimeUnit.SECONDS.sleep(10);
        }
    }

run this code for 5 mins, then using jconsole perform full GC and in thread list you can see left hanging threads even after webdriver session was closed and enough time passed to finish all threads. In production env I can see such threads hanging for days. Same issue is observable on selenium 4.8.3, devtools-v111, chromedriver 111.0.5563.64 Stacktrace of thread is:
Name: HttpClient-20-SelectorManager
State: RUNNABLE
Total blocked: 0  Total waited: 0

Stack trace: 
[email protected]/sun.nio.ch.epoll.wait(Native Method)
[email protected]/sun.nio.ch.epollselectorimpl.doselect(epollselectorimpl.java:118)
[email protected]/sun.nio.ch.selectorimpl.lockanddoselect(selectorimpl.java:129)
   - locked sun.nio.ch.Util$2@395d80e0
   - locked sun.nio.ch.EPollSelectorImpl@3fb5076d
[email protected]/sun.nio.ch.selectorimpl.select(selectorimpl.java:141)
platform/[email protected]/jdk.internal.net.http.httpclientimpl$selectormanager.run(httpclientimpl.java:889)

hanging connections: 
netstat -an | grep gridip
tcp        0      0 myip     gridip:4444       ESTABLISHED
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       1      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       1      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT 
tcp6       0      0 myip     gridip:4444       CLOSE_WAIT

Relevant log output

no log output required

Operating System

Debian 9, Ubuntu 20.04.3

Selenium version

Java 4.8.0, dev tools v109

What are the browser(s) and version(s) where you see this issue?

chrome 109.0

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 109.0.5414.74

Are you using Selenium Grid?

Yes, 4.8.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-javaJava BindingsI-defectSomething is not working as intended

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions