Skip to content

tests/*: pexpect: always expect before sendline#9220

Closed
ZetaR60 wants to merge 2 commits intoRIOT-OS:masterfrom
ZetaR60:RIOT_pexpect_wait
Closed

tests/*: pexpect: always expect before sendline#9220
ZetaR60 wants to merge 2 commits intoRIOT-OS:masterfrom
ZetaR60:RIOT_pexpect_wait

Conversation

@ZetaR60
Copy link
Copy Markdown
Contributor

@ZetaR60 ZetaR60 commented May 29, 2018

This is my attempt at fixing problems with the Murdock board testing. It intermittently fails on a few of the tests:

--- run_test job results (3 failed, 71 passed, 74 total):
    failed:
    tests/ps_schedstatistics/samr21-xpro
    tests/struct_tm_utility/samr21-xpro
    tests/shell/samr21-xpro

    passed:
    tests/gnrc_ipv6_nib_6ln/samr21-xpro
    tests/xtimer_now64_continuity/samr21-xpro
    tests/buttons/samr21-xpro
    tests/thread_msg_block_wo_queue/samr21-xpro
    tests/rmutex/samr21-xpro
    tests/posix_time/samr21-xpro
    tests/sched_testing/samr21-xpro
    tests/evtimer_msg/samr21-xpro
    tests/sizeof_tcb/samr21-xpro
    tests/pkg_micro-ecc/samr21-xpro
    tests/pthread/samr21-xpro
    tests/xtimer_periodic_wakeup/samr21-xpro
    tests/trickle/samr21-xpro
    tests/xtimer_msg/samr21-xpro
    tests/pthread_tls/samr21-xpro
    tests/cb_mux/samr21-xpro
    tests/gnrc_ndp/samr21-xpro
    tests/thread_msg/samr21-xpro
    tests/isr_yield_higher/samr21-xpro
    tests/lwip_sock_ip/samr21-xpro
    tests/xtimer_hang/samr21-xpro
    tests/thread_msg_block_w_queue/samr21-xpro
    tests/posix_semaphore/samr21-xpro
    tests/nhdp/samr21-xpro
    tests/thread_flags_xtimer/samr21-xpro
    tests/xtimer_reset/samr21-xpro
    tests/thread_flags/samr21-xpro
    tests/gnrc_ipv6_nib/samr21-xpro
    tests/cpp11_condition_variable/samr21-xpro
    tests/gnrc_sock_udp/samr21-xpro
    tests/msg_avail/samr21-xpro
    tests/evtimer_underflow/samr21-xpro
    tests/netdev_test/samr21-xpro
    tests/thread_flood/samr21-xpro
    tests/xtimer_usleep_short/samr21-xpro
    tests/periph_timer/samr21-xpro
    tests/fmt_print/samr21-xpro
    tests/xtimer_remove/samr21-xpro
    tests/pthread_cleanup/samr21-xpro
    tests/pthread_barrier/samr21-xpro
    tests/mutex_unlock_and_sleep/samr21-xpro
    tests/pkg_libcoap/samr21-xpro
    tests/events/samr21-xpro
    tests/irq/samr21-xpro
    tests/bloom_bytes/samr21-xpro
    tests/xtimer_msg_receive_timeout/samr21-xpro
    tests/xtimer_usleep/samr21-xpro
    tests/thread_race/samr21-xpro
    tests/pkg_cayenne-lpp/samr21-xpro
    tests/mutex_order/samr21-xpro
    tests/pkg_tinycbor/samr21-xpro
    tests/cpp11_thread/samr21-xpro
    tests/float/samr21-xpro
    tests/pkg_jsmn/samr21-xpro
    tests/thread_cooperation/samr21-xpro
    tests/pthread_cooperation/samr21-xpro
    tests/ssp/samr21-xpro
    tests/pkg_tiny-asn1/samr21-xpro
    tests/thread_exit/samr21-xpro
    tests/pkg_minmea/samr21-xpro
    tests/msg_try_receive/samr21-xpro
    tests/pthread_condition_variable/samr21-xpro
    tests/pipe/samr21-xpro
    tests/thread_basic/samr21-xpro
    tests/msg_send_receive/samr21-xpro
    tests/bitarithm_timings/samr21-xpro
    tests/cb_mux_bench/samr21-xpro
    tests/od/samr21-xpro
    tests/pkg_umorse/samr21-xpro
    tests/cpp11_mutex/samr21-xpro
    tests/gnrc_sock_ip/samr21-xpro

Those tests that it fails on are exactly those in which the python test script is sending using pexpect as well as receiving. It is intermittent as well, which suggests timing issues with the commands being issued. I have changed the scripts to ensure that before every send, it waits for the > shell prompt.

@aabadie aabadie added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels May 29, 2018
@ZetaR60 ZetaR60 mentioned this pull request May 29, 2018
Merged
@aabadie aabadie added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels May 29, 2018
@aabadie
Copy link
Copy Markdown
Contributor

aabadie commented May 29, 2018

@ZetaR60, I triggered Murdock of couple of times, it passed the first time but now it fails again.

@bergzand
Copy link
Copy Markdown
Member

bergzand commented May 29, 2018

@ZetaR60 This is an issue with the current CI hardware tests. Mostly caused by flaky serial connections to the boards.

@kaspar030
Copy link
Copy Markdown
Contributor

kaspar030 commented May 29, 2018

@ZetaR60 This is an issue with the current CI hardware tests. Mostly caused by flaky serial connections to the boards.

I think it is caused by the Linux drivers. Usually, a reboot of the raspi makes it work again for a while. ;(

@kYc0o
Copy link
Copy Markdown
Contributor

kYc0o commented May 29, 2018

I tried to address this issues in the same way as you but we decided that we would tackle this in a different way. I'd suggest to close it and look for a different solution.

@ZetaR60
Copy link
Copy Markdown
Contributor Author

ZetaR60 commented May 29, 2018

Yeah. It is pretty clear that it thinks it is sending one thing, but the board is getting something else.

@ZetaR60 ZetaR60 closed this May 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ZetaR60 @aabadie @bergzand @kaspar030 @kYc0o