I'm using a htmlunit (version 3) to scrapping a webpage.

HtmlPage p = webClient.getPage(URL);
HtmlElement b = p.getFirstByXPath("xpath selector1");
HtmlPage a = b.click();
a.executeJavaScript("window.scrollTo(0, 6000);");
webClient.waitForBackgroundJavaScript(2000);

My requirement is when open a page I have to do a click on a filtering button. The data is updated but the page is lazy loading, so I need to simulate scrolling to the bottom of the page to get more items. When a scroll is done a background request is executed by the server and retrieve more data.

I have a WebConnectionWrapper where catch the background request/response.

But an 400 Bad Request for URL error appear when I add this line on my code because I need to wait for a javascript scroll execution to listening the request that response with new data.

webClient.waitForBackgroundJavaScript(2000);

And without this line the request is done without any problem.

What could be the problem when add a waitForBackgroundJavaScript() ?