-
-
Notifications
You must be signed in to change notification settings - Fork 184
Description
I'm trying to navigate through the page and get the itens from the table wich is dynamically loaded.
When I see the page's XML the items don't exist, however when analyzing the page's code in the browser, the items are there.
My code:
` WebClient client = new WebClient(BrowserVersion.BEST_SUPPORTED);
CookieManager cookieManager = client.getCookieManager();
cookieManager.setCookiesEnabled(true);
client.setCookieManager(cookieManager);
client.getOptions().setPopupBlockerEnabled(true);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setCssEnabled(false);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setRedirectEnabled(true);
client.setAjaxController(new NicelyResynchronizingAjaxController());
client.getOptions().setThrowExceptionOnScriptError(false);
client.waitForBackgroundJavaScriptStartingBefore(50000);
//Deixa a pagina em portugues
WebRequest request = new WebRequest(new URL("https://portaldecompras.firjan.com.br/portal/Mural.aspx"), HttpMethod.POST);
request.setAdditionalHeader("Accept-Language", "pt-BR");
try {
//Pega URL do portal
HtmlPage paginaConsulta2 = client.getPage(vUrlPortal);
LOGGER.info(paginaConsulta2.asXml());
...........
`
XML result:
</thead> <tbody id="trListaMuralProcesso"/> </table> <div id="carregandoLista" class="carregandoDados" style="display:none" ;="">
Browser html result:
<tbody id="trListaMuralProcesso"> <tr> <td width="6%">3277</td> <td width="11%" onclick='trListaMuralProcesso_Click(3277, 18,"",false)' class="areaClique">PE 106/2022</td> <td width="10%">SERVIÇO SOCIAL DA INDUSTRIA - SESI - SES_CINELANDIA</td> <td width="22%">Aquisição de nobreaks, em atendimento à Firjan, ao SENAI e ao SESI.</td> <td width="13%">Pregão eletrônico</td> <td width="17%">30/08/2022 11:00</td> <td width="17%">09/09/2022 09:30</td> <td width="2%"> <a class="tooltip" data-title="Agendado/Publicado"><img src="../core/images/icones/ic_P_agendado.gif" /></a> <span class="areaClique" onclick="trListaMuralResumoEdital_Click(3277, 18, 3277, true)"> <a class="tooltip" data-title="Resumo do edital"> <img class="img-ponteiro" src="/Portal/Imagens/icon_edital_azul_16.png" /> </a> </span> </td> </tr> <tr> <td width="6%">3276</td> <td width="11%" onclick='trListaMuralProcesso_Click(3276, 18,"",false)' class="areaClique">PE 115/2022</td> <td width="10%">SERVIÇO NACIONAL DE APRENDIZAGEM INDUSTRIAL - SENAI - SEN_CTS_AUTO</td> <td width="22%">Aquisição de materiais elétricos em atendimento ao SENAI e SESI.</td> <td width="13%">Pregão eletrônico</td> <td width="17%">30/08/2022 10:00</td> <td width="17%">08/09/2022 13:30</td> <td width="2%"> <a class="tooltip" data-title="Em proposta"><img src="../core/images/icones/ic_P_em_proposta.gif" /></a> <span class="areaClique" onclick="trListaMuralResumoEdital_Click(3276, 18, 3276, true)"> <a class="tooltip" data-title="Resumo do edital"> <img class="img-ponteiro" src="/Portal/Imagens/icon_edital_azul_16.png" /> </a> </span> </td> </tr> .............
I realized this table loads from a javascript function which is written inside the page's file, and when HTMLUNIT tries to open it, it doesn't find a function. The function that loads the table is PesquisarProcessos and the JS where its written stays inside a div.
JS Function inside the page:
`<div id="corpo">
<div class="conteudo">
<!-- Scritps -->
<script type="text/javascript">
//<![CDATA[
.............................................
function OcultarFiltroPorModulo() {........................ }
function PesquisarSituacoesModulo(_nCdModulo) {......................... }
function PesquisarTipoProcesso() {...................}
function menuFiltro_MouseEnter() {....................}
function menuFiltro_MouseLeave() {....................}
function menuOpcoes_MouseEnter() {...................}
function menuOpcoes_MouseLeave() {.................. }
function ChegouFinalScroll() {.................}
function trListaMuralProcesso_Click(_nCdProcesso, _nCdModulo, _sDsProcesso, bAbreItemAutomatico, _nCdEdital = -2147483648) {.....................}
function trListaMuralResumoEdital_Click(_nCdProcesso, _nCdModulo, _nCdOrigem, _exibeBotaoDetalhes) {............}
function btProcurar_Click() {.................
}
function btOrdenarPor_Click(_sColuna, _oElemento) {.............
}
var bFlExibepercenteconomia = "-2147483647";
var bFlExibevleconomia = "-2147483647";
var bFlExibevlestimado = "-2147483647";
function **PesquisarProcessos**(_bProcurar) {.....................}`
The HTMLUNIT doesn't seem to find the function and return the error
`022-08-30 10:34:10.374 ERROR 8096 --- [pool-2-thread-1] c.g.h.j.DefaultJavaScriptErrorListener : Error during JavaScript execution
com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: "PesquisarProcessos" is not defined.
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:954)`
Is it possible to solve this problem?