This issue lists the status of all tests for the Release Candidate 2 of the 2019.01 release.

Specs tested:

• 01-ci
  • Task #01 - Compile test
  • Task #02 - Unittests on native @aabadie
  • Task #03 - Unittests on native separated @aabadie
  • Task #04 - Unittests on iotlab-m3 @aabadie
• 02-tests @aabadie
  • Task #01 - All tests on native
  • Task #02 - Subset of tests on iotlab-m3
  • Task #03 - Subset of tests on samr21-xpro
• 03-single-hop-ipv6-icmp
  • Task #01 - ICMPv6 multicast echo on native @aabadie
  • Task #02 - ICMPv6 link-local echo on native @aabadie
  • Task #03 - ICMPv6 link-local echo on native (1 hour)
  • Task #04 - ICMPv6 stress test on native (1 hour)
• 04-single-hop-6lowpan-icmp @aabadie
  • Task #01 - ICMPv6 link-local echo with iotlab-m3
  • Task #02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro
  • Task #03 - ICMPv6 echo with large payload
  • Task #04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes
  • Task #05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/remote
  • Task #06 (Experimental) - ICMPv6 link-local echo with samr21-xpro/remote
  • Task #07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee
  • Task #08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee
  • Task #09 - ICMPv6 stress test on iotlab-m3
    
• 05-single-hop-route @aabadie
  • Task #01 - ICMPv6 echo unicast addresess on native (default route)
  • Task #02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route)
  • Task #03 - ICMPv6 echo unicast addresess on native (specific route)
  • Task #04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route)
• 06-single-hop-udp
  • Task #01 - UDP on iotlab-m3
  • Task #02 - UDP on iotlab-m3 (UDP port compression)
  • Task #03 - UDP on native (non-existent neighbor)
  • Task #04 - UDP on iotlab-m3 (non-existent neighbor)
• 07-multi-hop
  • Task #01 - ICMPv6 echo on iotlab-m3 with three hops (static route)
  • Task #02 - UDP on iotlab-m3 with three hops (static route)
  • Task #03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route)
  • Task #04 - UDP on iotlab-m3 with three hops (RPL route)
• 08-interop
  • Task #01 - ICMPv6 echo between native and Linux
  • Task #02 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN
  • Task #03 - ICMPv6 echo between iotlab-m3 and Contiki (@miri64)
  • Task #04 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN (@miri64)
  • Task #05 - ICMPv6 echo between iotlab-m3 and Internet host through RIOT border router (@miri64)
  • Task #06 - UDP between iotlab-m3 and Internet host through RIOT border router
  • Task #07 - UDP between iotlab-m3 and Internet host through RIOT border router (200b payload)
  • Task #08 - UDP between GNRC and lwIP on iotlab-m3
  • Task #09 - UDP between GNRC and emb6 on iotlab-m3
  • Task #10 - UDP between lwIP and emb6 on iotlab-m3
• 09-coap @kb2ma
  • Task #01 - CORD Endpoint
  • Task #02 - Confirmable retries
  • Task #03 - Block1
  • Task #04 - Block2
  • Task #05 - Observe registration and notification
• 10-icmpv6-error @miri64
  • Task #01 - Destination unreachable - no route to destination
  • Task #02 - Destination unreachable - Beyond scope of source address
  • Task #03 - Destination unreachable - address unreachable (target node address)
  • Task #04 - Destination unreachable - address unreachable (neighbor cache miss)
  • Task #05 - Destination unreachable - port unreachable
  • Task #06 - Destination unreachable - port unreachable (large payload)
  • Task #07 - Packet too big
  • Task #08 - Time exceeded - hop limit exceeded in transit
  • Task #09 - Parameter problem - erroneous header field encountered
  • Task #10 - IPv6-in-IPv6 encapsulation
• 99-compile-and-test-one-board
  • Task #01 - Run tests on different hardwares
    • iotlab-m3 with BUILD_IN_DOCKER=1 same as normal + failed compilation for tests/riotboot
    • arduino-mega2560 I will list issues and details below Full errors list: Release 2019.01 - RC2 #98 (comment)
      • Firmware failing
        • tests/bitarithm_timings/ : firmware does not go past the bitarithm_msb: 102096 iterations per second
        • tests/periph_eeprom: EEPROM_CLEAR_BYTE api not handled in the test tests/periph_eeprom: use EEPROM_CLEAR_BYTE RIOT#11005
        • tests/periph_gpio: both firmware and test script I think. Some values look like to overflow 4294930872us and a timeout in the test for getting one line 02-bench.py:22 for bench 0 4 it works with a 30 seconds timeout for expect at least.
        • tests/pipe: the output is completely wrong and mixed
        • tests/pkg_jsmn: output is wrong no value looks like to be read
        • tests/pkg_libb2: blake2_tests.test_blake2s (/data/riotbuild/riotproject/tests/pkg_libb2/main.c 56) memcmp(b2s, hash, sizeof hash) == 0
        • tests/pkg_lora-serialization:

          2019-02-12 12:33:16,709 - INFO # Test 2
          2019-02-12 12:33:16,734 - INFO # Coordinates and unix time
          2019-02-12 12:33:16,771 - INFO # ---------------------------------
          2019-02-12 12:33:16,816 - INFO # - Writing coordinates: -33.905052, 151.26641
          2019-02-12 12:33:16,849 - INFO # - Writing unix time: 1467632413
          2019-02-12 12:33:16,902 - INFO # - Encoded:  64 a6 ff ff 60 24 00 00 1d 4b 00 00
          2019-02-12 12:33:16,951 - INFO # - Expected: 64 a6 fa fd 6a 24 04 09 1d 4b 7a 57
          2019-02-12 12:33:16,984 - INFO # ---------------------------------
          2019-02-12 12:33:16,992 - INFO # FAILED
          

        • tests/posix_semaphore:

          2019-02-12 12:35:09,967 - INFO # ######################### TEST4:
          2019-02-12 12:35:09,983 - INFO # first: sem_init s1
          2019-02-12 12:35:10,012 - INFO # first: wait 1 sec for s1
          2019-02-12 12:35:14,026 - INFO # first: timed out
          2019-02-12 12:35:14,075 - INFO # first: waited too long 3999748 usec => FAILED
          2019-02-12 12:35:14,108 - INFO # ######################### DONE
          

        • tests/pkg_micro-ecc: fails in the middle but micro-ecc does not have 8bit/16bit support.
        • tests/pkg_tiny-asn1: ERROR: Could not allocate the memory for the ASN.1 objects.
        • tests/ps_schedstatistics: the main thread cannot read stdin changing threads priority to a lower one (THREAD_PRIORITY_MAIN + 1) makes the test pass. (BTW test is wrongly written as | has a meaning for pexpect).
        • tests/rng: float printing issue: tests/rng: replace printf_float with fmt/print_float RIOT#10999
        • tests/trickle: it stops after [TRICKLE RESET] (known issue I think)
      • Test script failing
        • tests/evtimer_msg/: margin is not big enough for the platform At 770 ms received msg 0: "supposed to be 659"
      • Non reproducible error:
        • tests/isr_yield_higher: firmware was printing TEST FAILED but I cannot reproduce anymore
      • Non checked issue
        • tests/libfixmath: output is flowing correctly. I see that there is output with letters 2.H532 but not sure if it is the issue. Not checked yet.