Skip to content

HtmlUnit doesn't load and execute javascript correctly #499

@Rhaniel

Description

@Rhaniel

I'm trying to navigate through the page and get the itens from the table wich is dynamically loaded.

When I see the page's XML the items don't exist, however when analyzing the page's code in the browser, the items are there.

My code:

  ` WebClient client = new WebClient(BrowserVersion.BEST_SUPPORTED);

    CookieManager cookieManager = client.getCookieManager();
    cookieManager.setCookiesEnabled(true);
    client.setCookieManager(cookieManager);

    client.getOptions().setPopupBlockerEnabled(true);
    client.getOptions().setThrowExceptionOnFailingStatusCode(false);
    client.getOptions().setJavaScriptEnabled(true);
    client.getOptions().setCssEnabled(false);
    client.getOptions().setUseInsecureSSL(true);
    client.getOptions().setRedirectEnabled(true);
    client.setAjaxController(new NicelyResynchronizingAjaxController());
    client.getOptions().setThrowExceptionOnScriptError(false);
    client.waitForBackgroundJavaScriptStartingBefore(50000);

    //Deixa a pagina em portugues
    WebRequest request = new WebRequest(new URL("https://portaldecompras.firjan.com.br/portal/Mural.aspx"), HttpMethod.POST);
    request.setAdditionalHeader("Accept-Language", "pt-BR");

    try {

        //Pega URL do portal
        HtmlPage paginaConsulta2 = client.getPage(vUrlPortal);

        LOGGER.info(paginaConsulta2.asXml());
        ...........
        `

XML result:
</thead> <tbody id="trListaMuralProcesso"/> </table> <div id="carregandoLista" class="carregandoDados" style="display:none" ;="">

Browser html result:
<tbody id="trListaMuralProcesso"> <tr> <td width="6%">3277</td> <td width="11%" onclick='trListaMuralProcesso_Click(3277, 18,"",false)' class="areaClique">PE 106/2022</td> <td width="10%">SERVIÇO SOCIAL DA INDUSTRIA - SESI - SES_CINELANDIA</td> <td width="22%">Aquisição de nobreaks, em atendimento à Firjan, ao SENAI e ao SESI.</td> <td width="13%">Pregão eletrônico</td> <td width="17%">30/08/2022 11:00</td> <td width="17%">09/09/2022 09:30</td> <td width="2%"> <a class="tooltip" data-title="Agendado/Publicado"><img src="../core/images/icones/ic_P_agendado.gif" /></a> <span class="areaClique" onclick="trListaMuralResumoEdital_Click(3277, 18, 3277, true)"> <a class="tooltip" data-title="Resumo do edital"> <img class="img-ponteiro" src="/Portal/Imagens/icon_edital_azul_16.png" /> </a> </span> </td> </tr> <tr> <td width="6%">3276</td> <td width="11%" onclick='trListaMuralProcesso_Click(3276, 18,"",false)' class="areaClique">PE 115/2022</td> <td width="10%">SERVIÇO NACIONAL DE APRENDIZAGEM INDUSTRIAL - SENAI - SEN_CTS_AUTO</td> <td width="22%">Aquisição de materiais elétricos em atendimento ao SENAI e SESI.</td> <td width="13%">Pregão eletrônico</td> <td width="17%">30/08/2022 10:00</td> <td width="17%">08/09/2022 13:30</td> <td width="2%"> <a class="tooltip" data-title="Em proposta"><img src="../core/images/icones/ic_P_em_proposta.gif" /></a> <span class="areaClique" onclick="trListaMuralResumoEdital_Click(3276, 18, 3276, true)"> <a class="tooltip" data-title="Resumo do edital"> <img class="img-ponteiro" src="/Portal/Imagens/icon_edital_azul_16.png" /> </a> </span> </td> </tr> .............

I realized this table loads from a javascript function which is written inside the page's file, and when HTMLUNIT tries to open it, it doesn't find a function. The function that loads the table is PesquisarProcessos and the JS where its written stays inside a div.

JS Function inside the page:

       `<div id="corpo">
                <div class="conteudo">
                    <!-- Scritps -->
                    <script type="text/javascript">
                        //<![CDATA[
                       .............................................
                        function OcultarFiltroPorModulo() {........................ }

                        function PesquisarSituacoesModulo(_nCdModulo) {......................... }

                        function PesquisarTipoProcesso() {...................}

                        function menuFiltro_MouseEnter() {....................}

                        function menuFiltro_MouseLeave() {....................}

                        function menuOpcoes_MouseEnter() {...................}

                        function menuOpcoes_MouseLeave() {.................. }

                        function ChegouFinalScroll() {.................}

                        function trListaMuralProcesso_Click(_nCdProcesso, _nCdModulo, _sDsProcesso, bAbreItemAutomatico, _nCdEdital = -2147483648) {.....................}

                        function trListaMuralResumoEdital_Click(_nCdProcesso, _nCdModulo, _nCdOrigem, _exibeBotaoDetalhes) {............}

                        function btProcurar_Click() {.................
                        }

                        function btOrdenarPor_Click(_sColuna, _oElemento) {.............
                        }

                        var bFlExibepercenteconomia = "-2147483647";
                        var bFlExibevleconomia = "-2147483647";
                        var bFlExibevlestimado = "-2147483647";
                        function **PesquisarProcessos**(_bProcurar) {.....................}`

The HTMLUNIT doesn't seem to find the function and return the error

`022-08-30 10:34:10.374 ERROR 8096 --- [pool-2-thread-1] c.g.h.j.DefaultJavaScriptErrorListener : Error during JavaScript execution

com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: "PesquisarProcessos" is not defined.
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:954)`

Is it possible to solve this problem?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions