pkg/openwsn: re-integrate the network stack as a package by fjmolinas · Pull Request #13824 · RIOT-OS/RIOT

fjmolinas · 2020-04-06T18:48:20Z

Contribution description

This is meant as a first step in re-intregrating OpenWSN network stack as a pkg to RIOT. It is heavily based on previous work by @PeterKietzmann, @MichelRottleuthner, @jia200x and others in #8570 and its successors.

Its currently tagged as WIP since there are still some cleanups to be done and more testing is needed but it is working on iotlab-m3, samr21-xpro, iotlab-a8-m3. I currently have some issues with cc2538 which is why it hasn't been included in the PR, same goes for at86rf215, but I really want to support openmote-b so I hope to get these sorted out for this pr.

The goal of this PR is to integrate OpenWSN and all is network security features. Because of the current state of the code, this means including up to the COAP (+ UDP, IPv6, RPL, 6TiSCH). Future work plans on isolating as much as the package as possible alongside the current refactoring that is going on in https://github.com/openwsn-berkeley/openwsn-fw. Please see the pkg documentation for more information.

This PR does not include the mac isolation work done by @jia200x with OpenWSN. But I'll be happy to include it or work on including it in a follow-up. I tried to layout the makefiles that would allow not including not needed modules in the future, but will probably need some adaptation.

There is currently a verbatim copy of some of OpenWSN hardware abstraction tests. These still require some cleanup if it makes sense to include them. A tests for the radio abstraction is missing.

Implementation Status

Please go through the pkg documentation, but as a summary:

Takes the whole OpenWSN stack up to COAP, but COAP is only needed for joining the network https://datatracker.ietf.org/doc/draft-ietf-6tisch-dtsecurity-zerotouch-join/
Tested on arm boards with a 32768Hz RTT supporting an at86rf2xx radio.
Requires openvisualizer

Missing from this PR:

Follow up PR to add OpenWSN sock
mac layer isolation done in pkg/openwsn: re-integrate the network stack as a package #8570 successors (not PR'd)
Complete the support of OpenWSN default HW (openmote-b) (issues with cc2538 and at86rf215)
Add openvisualizer in dist/tools
Support for other 802.15.4 network drivers than at86rf2xx

Todo:

sctimer to trigger an ISR immediately we assume that RTT_IRQ is defined
and mapped to the matching IRQn_Type. This is no consistent across all
platforms, not even against all arm platforms.
RTT_FREQUENCY is not configurable for most platforms, implementations should
be adapted to make this configurable or their defaults changed to 32Khz.
Generalize board_info.h to other RTT_FREQUENCY.
Merge support for at86rf2xx basic mode at86rf2xx: implement basic mode (v2) #13798.
Merge ccms support sys/crypto: add ccms #13149.
Generic naming cleanups.

Testing

Refer to the main test application documentation in tests/pkg_openwsn.

Otherwise I still need to do more serious testing.

Missing documentatio? Feedback on the approach?

Issues/PRs references

Depends on ~~#13798~~ ~~#13149~~
Successor of #8570

Related to #12910
Partially addresses #13259

pkg/openwsn/contrib/radio.c

fjmolinas · 2020-04-20T07:38:02Z

I realized that stm32f1 rtt is not recommended to operate at higher frequencies than 16k, I added a timer division functionality between sctimer and rtt that should work for frequencies of the type 32768/2**n. Also removed some of the already merged dependencies.

fjmolinas · 2020-04-20T13:08:12Z

829cca5 will be removed with #13907.

fjmolinas · 2020-04-23T12:23:41Z

I would like #13878 instead of bc3b543, but I don't want to have this PR depend on this.

fjmolinas · 2020-05-04T15:14:31Z

This PR is no longer WIP, but still waiting for some PR's, currently some ztimer changes.

fjmolinas · 2020-05-04T15:15:58Z

I would like #13878 instead of bc3b543, but I don't want to have this PR depend on this.

For now I removed this change from this PR, and will tackle that in parallel, the stack still works without crc info, but shows some debug errors when using openvisualizer. The doc has been updated as well.

sctimer to trigger an ISR immediately we assume that RTT_IRQ is defined
and mapped to the matching IRQn_Type. This is no consistent across all
platforms, not even against all arm platforms.
RTT_FREQUENCY is not configurable for most platforms, implementations should
be adapted to make this configurable or their defaults changed to 32Khz.

I'm now using ztimer to addeess this.

fjmolinas · 2020-05-07T08:59:11Z

@TimothyClaeys might be interested as well.

fjmolinas · 2020-06-29T07:45:41Z

@aabadie I added in c19794a a small message on what goes on with OpenVisualizer over Iot-LAB

aabadie

I found some other minor things in the application README and code.

Otherwise, I was able to make it work on IoT-LAB, between 2 iotlab-m3 as leaves and 1 iotlab-m3 as root. It worked pretty well, not critical errors displayed in openvisualizer, the nodes were able to synchronize and exchange udp packets.

I also noticed that the shell sometimes "eats" some characters.

When the minor things are addressed, I think we can merge this PR. Good job @fjmolinas !

pkg/openwsn/contrib/debugpins.c

tests/pkg_openwsn/README.md

tests/pkg_openwsn/main.c

tests/pkg_openwsn/udp.c

fjmolinas · 2020-06-30T07:42:19Z

@aabadie addressed comments, may I squash?

pkg/openwsn/Makefile

aabadie · 2020-06-30T08:36:39Z

With the following cleanup in the build system, the build is still working:

diff --git a/pkg/openwsn/Makefile b/pkg/openwsn/Makefile
index 0ea448aaac..4680159f02 100644
--- a/pkg/openwsn/Makefile
+++ b/pkg/openwsn/Makefile
@@ -1,4 +1,4 @@
-PKG_NAME=openwsn-fw
+PKG_NAME=openwsn
 PKG_URL=https://github.com/openwsn-berkeley/openwsn-fw.git
 PKG_VERSION=cbcf622bd9369fcfc8455a5fb9349de2ed3c3a46
 PKG_LICENSE=BSD-3-Clause
@@ -19,7 +19,6 @@ OPENWSN_MODULES := $(filter-out $(IGNORE_MODULES),$(filter openwsn_%,$(USEMODULE
 .PHONY: openwsn_%
 
 all: $(OPENWSN_MODULES)
-       "$(MAKE)" -C $(PKG_SOURCE_DIR) -f $(CURDIR)/Makefile.$(PKG_NAME)
 
 OPENWSN_LOG_LEVEL ?= LOG_NONE
 OPENWSN_CFLAGS = -DLOG_LEVEL=$(OPENWSN_LOG_LEVEL)
diff --git a/pkg/openwsn/Makefile.openwsn-fw b/pkg/openwsn/Makefile.openwsn-fw
deleted file mode 100644
index 5e18220f81..0000000000
--- a/pkg/openwsn/Makefile.openwsn-fw
+++ /dev/null
@@ -1,5 +0,0 @@
-MODULE = openwsn
-
-NO_AUTO_SRC = 1
-
-include $(RIOTBASE)/Makefile.base

aabadie · 2020-06-30T08:48:36Z

Here is an improved version of the build system integration:

diff --git a/pkg/openwsn/Makefile b/pkg/openwsn/Makefile
index 0ea448aaac..944bf419bd 100644
--- a/pkg/openwsn/Makefile
+++ b/pkg/openwsn/Makefile
@@ -1,4 +1,4 @@
-PKG_NAME=openwsn-fw
+PKG_NAME=openwsn
 PKG_URL=https://github.com/openwsn-berkeley/openwsn-fw.git
 PKG_VERSION=cbcf622bd9369fcfc8455a5fb9349de2ed3c3a46
 PKG_LICENSE=BSD-3-Clause
@@ -18,45 +18,34 @@ OPENWSN_MODULES := $(filter-out $(IGNORE_MODULES),$(filter openwsn_%,$(USEMODULE
 
 .PHONY: openwsn_%
 
-all: $(OPENWSN_MODULES)
-       "$(MAKE)" -C $(PKG_SOURCE_DIR) -f $(CURDIR)/Makefile.$(PKG_NAME)
-
 OPENWSN_LOG_LEVEL ?= LOG_NONE
-OPENWSN_CFLAGS = -DLOG_LEVEL=$(OPENWSN_LOG_LEVEL)
-openwsn_%: CFLAGS += $(OPENWSN_CFLAGS)
-
-openwsn_openstack:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_openapps:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openapps -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_drivers:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/drivers/common -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_scheduler:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/kernel/openos -f $(CURDIR)/Makefile.openwsn_module
 
-openwsn_cjoin:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openapps/cjoin -f $(CURDIR)/Makefile.openwsn_module
+CFLAGS += -Wno-array-bounds
+CFLAGS += -Wno-implicit-fallthrough
+CFLAGS += -Wno-implicit-function-declaration
+CFLAGS += -Wno-incompatible-pointer-types
+CFLAGS += -Wno-maybe-uninitialized
+CFLAGS += -Wno-old-style-definition
+CFLAGS += -Wno-return-type
+CFLAGS += -Wno-sign-compare
+CFLAGS += -Wno-unused-parameter
+CFLAGS += -Wno-strict-prototypes
+CFLAGS += -DLOG_LEVEL=$(OPENWSN_LOG_LEVEL)
+
+OPENWSN_PATH_openstack      = openstack
+OPENWSN_PATH_openapps       = openapps
+OPENWSN_PATH_drivers        = drivers/common
+OPENWSN_PATH_scheduler      = kernel/openos
+OPENWSN_PATH_cjoin          = openapps/cjoin
+OPENWSN_PATH_opencoap       = openapps/opencoap
+OPENWSN_PATH_mac_low        = openstack/02a-MAClow
+OPENWSN_PATH_mac_high       = openstack/02b-MAChigh
+OPENWSN_PATH_iphc           = openstack/03a-IPHC
+OPENWSN_PATH_ipv6           = openstack/03b-IPv6
+OPENWSN_PATH_transport      = openstack/04-TRAN
+OPENWSN_PATH_crosslayers    = openstack/cross-layers
 
-openwsn_opencoap:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openapps/opencoap -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_mac_low:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/02a-MAClow -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_mac_high:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/02b-MAChigh -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_iphc:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/03a-IPHC -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_ipv6:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/03b-IPv6 -f $(CURDIR)/Makefile.openwsn_module
-
-openwsn_transport:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/04-TRAN -f $(CURDIR)/Makefile.openwsn_module
+all: $(OPENWSN_MODULES)
 
-openwsn_crosslayers:
-       MODULE=$@ "$(MAKE)" -C $(PKG_SOURCE_DIR)/openstack/cross-layers -f $(CURDIR)/Makefile.openwsn_module
+openwsn_%:
+       "$(MAKE)" -C $(PKG_SOURCE_DIR)/$(OPENWSN_PATH_$*) -f $(RIOTBASE)/Makefile.base MODULE=$@
diff --git a/pkg/openwsn/Makefile.openwsn-fw b/pkg/openwsn/Makefile.openwsn-fw
deleted file mode 100644
index 5e18220f81..0000000000
--- a/pkg/openwsn/Makefile.openwsn-fw
+++ /dev/null
@@ -1,5 +0,0 @@
-MODULE = openwsn
-
-NO_AUTO_SRC = 1
-
-include $(RIOTBASE)/Makefile.base
diff --git a/pkg/openwsn/Makefile.openwsn_module b/pkg/openwsn/Makefile.openwsn_module
deleted file mode 100644
index 66801744d6..0000000000
--- a/pkg/openwsn/Makefile.openwsn_module
+++ /dev/null
@@ -1,12 +0,0 @@
-CFLAGS += -Wno-array-bounds
-CFLAGS += -Wno-implicit-fallthrough
-CFLAGS += -Wno-implicit-function-declaration
-CFLAGS += -Wno-incompatible-pointer-types
-CFLAGS += -Wno-maybe-uninitialized
-CFLAGS += -Wno-old-style-definition
-CFLAGS += -Wno-return-type
-CFLAGS += -Wno-sign-compare
-CFLAGS += -Wno-unused-parameter
-CFLAGS += -Wno-strict-prototypes
-
-include $(RIOTBASE)/Makefile.base

aabadie

LGTM now. Thanks @fjmolinas for addressing the comments and improving the ifconfig output.

ACK, please squash !

Co-authored-by: Peter Kietzmann <[email protected]> Co-authored-by: Jose Alamos <[email protected]> Co-authored-by: Michel Rottleuthner <[email protected]>

Co-authored-by: Peter Kietzmann <[email protected]>

fjmolinas · 2020-06-30T11:12:34Z

@aabadie There was missing change after your makefile suggestions, they didn't show up initially because the old name for the pkg openwsn-fw was still cloned locally. This is fixed now and the openv-% commands are still good. Murdock should not complain this time.

BAUD=19200 make -C tests/pkg_openwsn/ openv-termroot PORT=/dev/riot/tty-ftdi
make: Entering directory '/home/francisco/workspace/RIOT3/tests/pkg_openwsn'

 ___                 _ _ _  ___  _ _ 
| . | ___  ___ ._ _ | | | |/ __>| \ |
| | || . \/ ._>| ' || | | |\__ \|   |
`___'|  _/\___.|_|_||__/_/ <___/|_\_|
     |_|                  openwsn.org

13:11:52 [OpenVisualizerServer:VERBOSE] Loading logging configuration: /home/francisco/workspace/RIOT3/tests/pkg_openwsn/bin/iotlab-m3/logging.conf
13:11:52 [OpenVisualizerServer:INFO] Initializing OV Server with options:
	- host address server     = localhost
	- port number server      = 9000
	- firmware path           = /home/francisco/workspace/RIOT3/build/pkg/openwsn
	- set root                = /dev/riot/tty-ftdi
	- use page zero           = False
	- use VCD logger          = False
	- serial port mask        = ['/dev/riot/tty-ftdi']
	- baudrates to probe      = ['19200']
13:11:52 [MoteProbe:WARNING] Probing motes: ['/dev/riot/tty-ftdi'] at baudrates ['19200']
13:11:52 [MoteProbe:SUCCESS] Discovered serial-port(s): ['/dev/riot/tty-ftdi']
13:11:52 [OpenVisualizerServer:INFO] Extracting firmware definitions.
13:11:52 [Utils:VERBOSE] Extracting firmware component names
13:11:52 [Utils:VERBOSE] Extracting firmware log descriptions.
13:11:52 [Utils:VERBOSE] Extracting 6top return codes.
13:11:52 [Utils:VERBOSE] Extracting 6top states.
13:11:52 [RPL:INFO] registering DAGroot 82-6b-de-ec-58-34-65-78
13:11:52 [OpenVisualizerServer:INFO] Setting DAG root...

aabadie · 2020-06-30T12:08:18Z

GO!

miri64 · 2020-06-30T12:08:49Z

🎉 Now who is making TSCH workable as a netif? ;-P

fjmolinas · 2020-06-30T12:12:11Z

Thanks everyone for all the time spend on reviewing (specially all the annoying typos I kept making) @aabadie @kaspar030 @PeterKietzmann!

fjmolinas added State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet Area: pkg Area: External package ports State: waiting for other PR State: The PR requires another PR to be merged first labels Apr 6, 2020

fjmolinas requested review from MichelRottleuthner, aabadie, daniel-k, jia200x, leandrolanzieri, miri64 and smlng as code owners April 6, 2020 18:48

fjmolinas mentioned this pull request Apr 7, 2020

at86rf2xx: implement basic mode (v2) #13798

Merged

PeterKietzmann reviewed Apr 8, 2020

View reviewed changes

pkg/openwsn/contrib/radio.c Outdated Show resolved Hide resolved

PeterKietzmann mentioned this pull request Apr 8, 2020

sys/crypto: add ccms #13149

Closed

This was referenced Apr 8, 2020

cc2420/gnrc_netif: Adds crc_valid to netif #8276

Closed

crc validity variable in netdev_radio_rx_info and gnrc_netif_hdr_t and read it for at86rf2xx and cc2420 #9755

Closed

fjmolinas force-pushed the pr_openwsn branch from 808c53d to 7f667be Compare April 20, 2020 07:34

fjmolinas force-pushed the pr_openwsn branch from 7f667be to 920b3da Compare May 4, 2020 15:13

fjmolinas requested review from bergzand and kaspar030 as code owners May 4, 2020 15:13

fjmolinas removed the State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet label May 4, 2020

fjmolinas force-pushed the pr_openwsn branch from 920b3da to 3aa1b49 Compare May 7, 2020 13:13

fjmolinas added Area: network Area: Networking Type: new feature The issue requests / The PR implemements a new feature for RIOT and removed State: waiting for other PR State: The PR requires another PR to be merged first labels May 7, 2020

aabadie requested changes Jun 29, 2020

View reviewed changes

aabadie reviewed Jun 30, 2020

View reviewed changes

pkg/openwsn/Makefile Outdated Show resolved Hide resolved

aabadie reviewed Jun 30, 2020

View reviewed changes

pkg/openwsn/Makefile Outdated Show resolved Hide resolved

aabadie reviewed Jun 30, 2020

View reviewed changes

pkg/openwsn/Makefile Outdated Show resolved Hide resolved

aabadie approved these changes Jun 30, 2020

View reviewed changes

fjmolinas force-pushed the pr_openwsn branch from a7ac395 to ba8735b Compare June 30, 2020 10:55

fjmolinas added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jun 30, 2020

fjmolinas and others added 2 commits June 30, 2020 13:08

pkg/OpenWSN: initial import

844ccfb

Co-authored-by: Peter Kietzmann <[email protected]> Co-authored-by: Jose Alamos <[email protected]> Co-authored-by: Michel Rottleuthner <[email protected]>

tests/pkg_openwsn: initial import

985a706

Co-authored-by: Peter Kietzmann <[email protected]>

fjmolinas added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jun 30, 2020

fjmolinas force-pushed the pr_openwsn branch from ba8735b to 59d82c0 Compare June 30, 2020 11:08

fjmolinas removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jun 30, 2020

fjmolinas added 3 commits June 30, 2020 13:10

dist/tools/openvisualizer: inital support

d01476a

boards/common/iotlab: add OpenWSN timing constants

7515f11

boards/samr21-xpro: add OpenWSN timing constants

b0c5c3e

fjmolinas added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jun 30, 2020

fjmolinas force-pushed the pr_openwsn branch from 59d82c0 to b0c5c3e Compare June 30, 2020 11:11

aabadie merged commit d98ddfa into RIOT-OS:master Jun 30, 2020

miri64 added this to the Release 2020.07 milestone Jun 30, 2020

fjmolinas deleted the pr_openwsn branch June 30, 2020 12:11

aabadie mentioned this pull request Jul 2, 2020

6TiSCH not supported in RIOT #13259

Closed

4 tasks

Conversation

fjmolinas commented Apr 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Implementation Status

Testing

Issues/PRs references

Uh oh!

Uh oh!

fjmolinas commented Apr 20, 2020

Uh oh!

fjmolinas commented Apr 20, 2020

Uh oh!

fjmolinas commented Apr 23, 2020

Uh oh!

fjmolinas commented May 4, 2020

Uh oh!

fjmolinas commented May 4, 2020

Uh oh!

fjmolinas commented May 7, 2020

Uh oh!

fjmolinas commented Jun 29, 2020

Uh oh!

aabadie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fjmolinas commented Jun 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aabadie commented Jun 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aabadie commented Jun 30, 2020

Uh oh!

aabadie left a comment

Choose a reason for hiding this comment

Uh oh!

fjmolinas commented Jun 30, 2020

Uh oh!

aabadie commented Jun 30, 2020

Uh oh!

miri64 commented Jun 30, 2020

Uh oh!

fjmolinas commented Jun 30, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

fjmolinas commented Apr 6, 2020 •

edited

Loading

aabadie commented Jun 30, 2020 •

edited

Loading