tests: add Nordic SoftDevice test app by haukepetersen · Pull Request #11797 · RIOT-OS/RIOT

haukepetersen · 2019-07-04T14:21:57Z

Contribution description

This is the next step towards making NimBLE the default IP-over-BLE solution for Nordic boards.

This PR adds a simple test application to verify that the Nordic SoftDevice package is still working. Though meant mainly for keeping the SoftDevice in the built-test, it also includes a (very shallow) test script, that simply checks if a device boots ok and that the SoftDevice registers a network device with GNRC.

Testing procedure

Build test should pass, and make test on any supported boards should succeed.

Issues/PRs references

loosely related to #11578

cladmi · 2019-07-05T13:21:12Z

It could be enabled on CI with TEST_ON_CI_WHITELIST += all as murdock has a nrf52dk connected.

dylad

Just two nitpicks, I'll try to test it in the coming days.

dylad · 2019-07-10T20:49:48Z

tests/nordic_softdevice/README.md

@@ -0,0 +1,13 @@
+# Nordic SoftDevice Test Application
+The main purpose of this test application is to ensure the inclusion of the
+Nodric SoftDevice package in RIOTs build test.


dylad · 2019-07-10T20:51:03Z

tests/nordic_softdevice/main.c

+ */
+
+/**
+ * @ingroup     examples


dylad · 2019-07-18T20:05:07Z

@haukepetersen tested manually on nrf52dk, it works fine but automated test seems broken. Nothing happens with make BOARD=nrf52dk -C tests/nordic_softdevice clean all flash test

miri64 · 2019-07-18T20:27:09Z

It does not echo, so no output means success ;)

dylad · 2019-07-18T20:54:58Z

@miri64 You're right I didn't notice.
In fact I tried on another test (i.e tests/thread_flags), and the whole output is print after Script processing completed.

$ make BOARD=nrf52dk -C tests/thread_flags clean all flash test
make : on entre dans le répertoire « /home/dylad/software/RIOT/tests/thread_flags »
Building application "tests_thread_flags" for "nrf52dk" with MCU "nrf52".

...


Script processing completed.

/home/dylad/software/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-07-18 22:45:18,790 - INFO # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
2019-07-18 22:45:22,095 - INFO # main(): This is RIOT! (Version: 2019.07-devel-941-gf1fc5-add_test_softdevice)
2019-07-18 22:45:22,096 - INFO # START
2019-07-18 22:45:22,098 - INFO # thread(): waiting for 0x1...
2019-07-18 22:45:22,101 - INFO # main(): setting flag 0x0001
2019-07-18 22:45:22,104 - INFO # thread(): received flags: 0x0001
2019-07-18 22:45:22,107 - INFO # thread(): waiting for 0x1 || 0x64...
2019-07-18 22:45:22,109 - INFO # main(): setting flag 0x0064
2019-07-18 22:45:22,112 - INFO # thread(): received flags: 0x0064
2019-07-18 22:45:22,115 - INFO # thread(): waiting for 0x2 && 0x4...
2019-07-18 22:45:22,118 - INFO # main(): setting flag 0x0001
2019-07-18 22:45:22,120 - INFO # main(): setting flag 0x0008
2019-07-18 22:45:22,122 - INFO # main(): setting flag 0x0002
2019-07-18 22:45:22,125 - INFO # main(): setting flag 0x0004
2019-07-18 22:45:22,128 - INFO # thread(): received flags: 0x0006
2019-07-18 22:45:22,132 - INFO # thread(): waiting for any flag, one by one
2019-07-18 22:45:22,134 - INFO # thread(): received flags: 0x0001
2019-07-18 22:45:22,138 - INFO # thread(): waiting for any flag, one by one
2019-07-18 22:45:22,141 - INFO # thread(): received flags: 0x0008
2019-07-18 22:45:22,144 - INFO # main: setting 100ms timeout...
2019-07-18 22:45:22,248 - INFO # main: timeout triggered. time passed: 100014us
2019-07-18 22:45:22,249 - INFO # SUCCESS

make : on quitte le répertoire « /home/dylad/software/RIOT/tests/thread_flags »

I don't have the same behavior with

$ make BOARD=nrf52dk -C tests/nordic_softdevice clean all flash test
...
Script processing completed.

make : on quitte le répertoire « /home/dylad/software/RIOT/tests/nordic_softdevice »

Am I missing something ?

cladmi · 2019-07-22T13:38:02Z

The script is configured to do run(..., echo=False) so there is never any output. This was done only in the gnrc_ scapy tests.

git grep 'echo=False'
gnrc_ipv6_ext/tests/01-run.py:    sys.exit(run(testfunc, timeout=1, echo=False))
gnrc_rpl_srh/tests/01-run.py:    sys.exit(run(testfunc, timeout=1, echo=False))
gnrc_sock_dns/tests/01-run.py:    sys.exit(run(testfunc, timeout=1, echo=False))

On success there is nothing printed.
And on error you only get an error message without knowing what happened (real output I got on first try):

Script processing completed.

Timeout in expect script at "child.expect(r"Iface\s+(\d+)\s+HWaddr:")" (tests/nordic_softdevice/tests/01-run.py:10)

/home/harter/work/git/RIOT/tests/nordic_softdevice/../../Makefile.include:604: recipe for target 'test' failed
make: *** [test] Error 1
make: Leaving directory '/home/harter/work/git/RIOT/tests/nordic_softdevice'

You can see the output by applying this patch:

diff --git a/tests/nordic_softdevice/tests/01-run.py b/tests/nordic_softdevice/tests/01-run.py
index 7329c7409..cda817d1d 100755
--- a/tests/nordic_softdevice/tests/01-run.py
+++ b/tests/nordic_softdevice/tests/01-run.py
@@ -11,4 +11,4 @@ def testfunc(child):


 if __name__ == "__main__":
-    sys.exit(run(testfunc, timeout=1, echo=False))
+    sys.exit(run(testfunc, timeout=1, echo=True))

Script processing completed.

/home/harter/work/git/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-07-22 15:33:57,901 - INFO # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
2019-07-22 15:34:01,623 - INFO # main(): This is RIOT! (Version: 2019.07-devel-941-gf1fc5-pr/riot/11797/add_Nordic_SoftDevice_test_app)
2019-07-22 15:34:01,628 - INFO # Test for the RIOT integration of the Nordic SoftDevice
2019-07-22 15:34:01,631 - INFO # All up, running the shell now
> ifconfig
2019-07-22 15:34:01,683 - INFO #  ifconfig
2019-07-22 15:34:01,687 - INFO # Iface  5  HWaddr: 00:FD:63:51:3C:6A

make: Leaving directory '/home/harter/work/git/RIOT/tests/nordic_softdevice'

The rest of ifconfig output is not printed as the script does not expect the end of the command with child.expect('>').

haukepetersen · 2019-07-31T15:24:57Z

@dylad Are your questions regarding the test target sufficiently answered?

I also fixed the typos that were pointed out, let me know if its ok to squash.

haukepetersen · 2019-07-31T15:29:29Z

also added the nrf52dk to the CI whitelist

dylad

ACK.
I am happy with this PR so you can squash.

haukepetersen · 2019-08-05T10:09:17Z

squashed and rebased.

haukepetersen · 2019-08-05T10:59:54Z

All green -> lets go!

kaspar030 · 2019-08-09T08:41:00Z

It seems this broke the nightlies.

miri64 · 2019-08-09T08:44:56Z

Mh... maybe would have been noticed earlier if RIOT-OS/murdock-scripts#24 were merged :-/

kaspar030 · 2019-08-09T08:47:14Z

Mh... maybe would have been noticed earlier if RIOT-OS/murdock-scripts#24 were merged :-/

I get the hint. ;)

blame mode on: the test was never run on CI before merging. Two "skipped automatic tests before merging/ACKing" points for both @haukepetersen and @dylad. 😉

haukepetersen added Area: tests Area: tests and testing framework Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: BLE Area: Bluetooth Low Energy support labels Jul 4, 2019

dylad requested changes Jul 10, 2019

View reviewed changes

haukepetersen force-pushed the add_test_softdevice branch from f1fc5fd to 7fdbb63 Compare July 31, 2019 15:23

dylad approved these changes Aug 2, 2019

View reviewed changes

tests: add Nordic SoftDevice test app

2e73e7f

haukepetersen force-pushed the add_test_softdevice branch from 38c34e0 to 2e73e7f Compare August 5, 2019 10:09

haukepetersen added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 5, 2019

haukepetersen merged commit a04d83e into RIOT-OS:master Aug 5, 2019

haukepetersen deleted the add_test_softdevice branch August 5, 2019 11:00

cladmi mentioned this pull request Aug 12, 2019

murdock: nordic_softdevice_ble: softdevice.hex is a test input #11995

Merged

kb2ma added this to the Release 2019.10 milestone Sep 16, 2019

Conversation

haukepetersen commented Jul 4, 2019

Contribution description

Testing procedure

Issues/PRs references

Uh oh!

cladmi commented Jul 5, 2019

Uh oh!

dylad left a comment

Choose a reason for hiding this comment

Uh oh!

dylad Jul 10, 2019

Choose a reason for hiding this comment

Uh oh!

dylad Jul 10, 2019

Choose a reason for hiding this comment

Uh oh!

dylad commented Jul 18, 2019

Uh oh!

miri64 commented Jul 18, 2019

Uh oh!

dylad commented Jul 18, 2019

Uh oh!

cladmi commented Jul 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haukepetersen commented Jul 31, 2019

Uh oh!

haukepetersen commented Jul 31, 2019

Uh oh!

dylad left a comment

Choose a reason for hiding this comment

Uh oh!

haukepetersen commented Aug 5, 2019

Uh oh!

haukepetersen commented Aug 5, 2019

Uh oh!

kaspar030 commented Aug 9, 2019

Uh oh!

miri64 commented Aug 9, 2019

Uh oh!

kaspar030 commented Aug 9, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

cladmi commented Jul 22, 2019 •

edited

Loading