SUIT: Update to draft-ietf-v3 by bergzand · Pull Request #13486 · RIOT-OS/RIOT

bergzand · 2020-02-26T14:09:23Z

Contribution description

This PR:

Removes the currently available draft-moran-v4 SUIT firmware update module.
Adds a reworked parser and update module based on draft-ietf-v3.
Adds tooling to generated v3-style manifests using ARM python code.
Adds a manifest parser test for the new SUIT firmware update module.
Reworks the examples/suit_update to use the new SUIT firmware update module.

Removal of moran-v4 parser

The moran-v4 parser is outdated by now and should be removed as a replacement is provided with this PR. It has always been marked as experimental, so (IMHO) it should not have to go through a deprecation period first.

SUIT draft-ietf-v3 parser

The new parser is based on version 3 of the ietf draft. Compared to the previous version the manifest format is simplified a bit and functionality provided between the two versions is similar. As it is a draft it again is marked experimental and we expect minor (but incompatible) changes in follow up versions.

Tooling

The tool to generate manifests can be used in a similar way as the v4 tools. The tool is based on python code provided by ARM.

Test application

The test application generates a key and a number of correct and incorrect manifests. These are parsed and the expected result is asserted.

For the v4 version I placed the generated manifests in bin/manifest (it is a binary file) and it conveniently ensures that the files are included in the gitignore, please let me know if this should also be done with this PR.

Testing procedure

Workflow as described by examples/suit_update works.
Test application passes.

Issues/PRs references

Obsoletes #13440

Co-authored-by: Kaspar Schleiser <[email protected]>

makefiles/suit.v3.base.inc.mk

sys/suit/v3/handlers.c

sys/suit/v3/handlers_common.c

sys/suit/v3/handlers_common_seq.c

sys/suit/v3/handlers_container.c

sys/suit/v3/handlers_global.c

examples/suit_update/Makefile

sys/include/suit/v3/handlers.h

sys/include/suit/v3/suit.h

fjmolinas · 2020-03-18T10:37:21Z

Here is a gist with putting up to date some things in the README. If you agree with the suggestions you can apply squash.

bergzand · 2020-03-18T12:57:53Z

Here is a gist with putting up to date some things in the README. If you agree with the suggestions you can apply squash.

Thanks! applied

fjmolinas · 2020-03-18T13:08:27Z

Thanks! applied

Pleas squash!

Co-authored-by: Kaspar Schleiser <[email protected]>

bergzand · 2020-03-19T09:06:59Z

Squashed!

fjmolinas · 2020-03-19T10:02:26Z

Seem there are some boards needing blacklist, and travis has a couple of complaints, Squash right away when addressed.

Otherwise there is still this:

@kaspar030 Is it possible that the Pi-based workers do not have ed25519 support in OpenSSL yet? The build run fails on generating the crypto keys.

bergzand · 2020-03-19T10:41:17Z

Seem there are some boards needing blacklist, and travis has a couple of complaints, Squash right away when addressed.

Added them

Co-authored-by: Kaspar Schleiser <[email protected]>

bergzand · 2020-03-20T08:15:45Z

@kaspar030, @fjmolinas All green here!

bergzand · 2020-03-20T08:37:53Z

@kaspar030, @fjmolinas All green here!

I added the examples/suit-update to the CI blacklist:

# This can be removed as soon as the Pi-fleet runners support an Openssl version
# with ed25519 support.
TEST_ON_CI_BLACKLIST = all

The suit_manifest test doesn't need to be blacklisted, all the crypto handling is done on build servers and those have a sufficiently modern Openssl.

fjmolinas

All my change requests have been addressed. I have tested the new test and the example. The Readme and documentation are up to date. ACK!

comments addressed

bergzand · 2020-03-20T13:48:36Z

🎉

Thanks!

benpicco · 2020-03-22T23:53:23Z

btw.: I now get

Traceback (most recent call last):
  File "/home/benpicco/dev/RIOT/dist/tools/suit_v3/suit-manifest-generator/bin/suit-tool", line 25, in <module>
    from suit_tool import clidriver
  File "/home/benpicco/dev/RIOT/dist/tools/suit_v3/suit-manifest-generator/bin/../suit_tool/clidriver.py", line 24, in <module>
    from suit_tool import create, sign, parse, get_uecc_pubkey
  File "/home/benpicco/dev/RIOT/dist/tools/suit_v3/suit-manifest-generator/bin/../suit_tool/create.py", line 20, in <module>
    from suit_tool.compile import compile_manifest
  File "/home/benpicco/dev/RIOT/dist/tools/suit_v3/suit-manifest-generator/bin/../suit_tool/compile.py", line 29, in <module>
    from suit_tool.manifest import SUITComponentId, SUITCommon, SUITSequence, \
  File "/home/benpicco/dev/RIOT/dist/tools/suit_v3/suit-manifest-generator/bin/../suit_tool/manifest.py", line 22, in <module>
    import cbor
ModuleNotFoundError: No module named 'cbor'

when building anything.
The build continues successfully, but this is of course annoying 😉

Maybe something like this?

diff --git a/Makefile.dep b/Makefile.dep
index 6c07ef3a2..4dc857eba 100644
--- a/Makefile.dep
+++ b/Makefile.dep
@@ -959,6 +959,9 @@ ifneq (,$(filter suit,$(USEMODULE)))
   USEMODULE += libcose_crypt_c25519
   USEMODULE += uuid
 
+  BUILDDEPS += $(SUIT_PUB_HDR)
+  CFLAGS += -I$(SUIT_PUB_HDR_DIR)
+
   # tests/suit_manifest has some mock implementations,
   # only add the non-mock dependencies if not building that test.
   ifeq (,$(filter suit_transport_mock,$(USEMODULE)))
diff --git a/makefiles/suit.base.inc.mk b/makefiles/suit.base.inc.mk
index ab0d84c10..710a3506e 100644
--- a/makefiles/suit.base.inc.mk
+++ b/makefiles/suit.base.inc.mk
@@ -21,8 +21,6 @@ SUIT_SEC ?= $(SUIT_KEY_DIR)/$(SUIT_KEY).pem
 
 SUIT_PUB_HDR = $(BINDIR)/riotbuild/public_key.h
 SUIT_PUB_HDR_DIR = $(dir $(SUIT_PUB_HDR))
-CFLAGS += -I$(SUIT_PUB_HDR_DIR)
-BUILDDEPS += $(SUIT_PUB_HDR)
 
 $(SUIT_SEC): $(CLEAN)
        @echo suit: generating key in $(SUIT_KEY_DIR)

suit: remove moran-v4 version of the parser

ef8daaf

Co-authored-by: Kaspar Schleiser <[email protected]>

bergzand requested review from aabadie, emmanuelsearch, fjmolinas and kaspar030 February 26, 2020 14:09

bergzand requested review from MichelRottleuthner, jia200x, leandrolanzieri, miri64 and smlng as code owners February 26, 2020 14:09

bergzand removed request for MichelRottleuthner, jia200x, leandrolanzieri, miri64 and smlng February 26, 2020 14:09

bergzand force-pushed the pr/suit/ietf_v3 branch from 47d4508 to 607d5f5 Compare February 26, 2020 15:03

bergzand added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Feb 26, 2020

kaspar030 reviewed Feb 26, 2020

View reviewed changes

makefiles/suit.v3.base.inc.mk Outdated Show resolved Hide resolved

bergzand removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Feb 27, 2020

kaspar030 previously requested changes Feb 27, 2020

View reviewed changes

makefiles/suit.v3.base.inc.mk Outdated Show resolved Hide resolved

bergzand added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Feb 27, 2020

bergzand mentioned this pull request Mar 3, 2020

tests/suit_v4_manifest: Add test for manifest parsing #13440

Closed

emmanuelsearch reviewed Mar 3, 2020

View reviewed changes

bergzand removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 4, 2020

bergzand and others added 2 commits March 18, 2020 14:13

dist/tools/suit_v3: Add ietf-v3 manifest generator

009a317

Co-authored-by: Kaspar Schleiser <[email protected]>

sys/suit: Add SUIT draft ietf-v3 firmware upgrade module

14bdf8f

Co-authored-by: Kaspar Schleiser <[email protected]>

bergzand force-pushed the pr/suit/ietf_v3 branch from 02eebb0 to 9f3bc27 Compare March 19, 2020 09:06

bergzand force-pushed the pr/suit/ietf_v3 branch from 9f3bc27 to 67e9279 Compare March 19, 2020 10:33

bergzand force-pushed the pr/suit/ietf_v3 branch from 67e9279 to 9f085e5 Compare March 19, 2020 12:58

bergzand removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 19, 2020

bergzand force-pushed the pr/suit/ietf_v3 branch from 9f085e5 to ed112a4 Compare March 19, 2020 20:15

bergzand added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 19, 2020

bergzand force-pushed the pr/suit/ietf_v3 branch from ed112a4 to 50ed7b0 Compare March 19, 2020 21:52

bergzand and others added 3 commits March 19, 2020 22:56

tests/suit_v3_manifest: Add draft-ietf-v3 manifest parser test

413b913

Co-authored-by: Kaspar Schleiser <[email protected]>

examples/suit: Refactor to draft ietf-v3

1d1d5b6

Co-authored-by: Kaspar Schleiser <[email protected]>

flake8: move exception from SUITv4 to SUITv3

829f592

Co-authored-by: Kaspar Schleiser <[email protected]>

bergzand force-pushed the pr/suit/ietf_v3 branch from 50ed7b0 to 829f592 Compare March 19, 2020 21:56

fjmolinas approved these changes Mar 20, 2020

View reviewed changes

kaspar030 merged commit 7d0c475 into RIOT-OS:master Mar 20, 2020

bergzand deleted the pr/suit/ietf_v3 branch March 20, 2020 13:48

benpicco mentioned this pull request Mar 22, 2020

suit-tool runs on every build - ModuleNotFoundError: No module named 'cbor' when building anything #13680

Closed

leandrolanzieri added this to the Release 2020.04 milestone Mar 26, 2020

aabadie mentioned this pull request Dec 30, 2020

sys/shell: fix missing suit shell command in examples/suit_update #15702

Merged

Conversation

bergzand commented Feb 26, 2020

Contribution description

Removal of moran-v4 parser

SUIT draft-ietf-v3 parser

Tooling

Test application

Testing procedure

Issues/PRs references

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fjmolinas commented Mar 18, 2020

Uh oh!

bergzand commented Mar 18, 2020

Uh oh!

fjmolinas commented Mar 18, 2020

Uh oh!

bergzand commented Mar 19, 2020

Uh oh!

fjmolinas commented Mar 19, 2020

Uh oh!

bergzand commented Mar 19, 2020

Uh oh!

bergzand commented Mar 20, 2020

Uh oh!

bergzand commented Mar 20, 2020

Uh oh!

fjmolinas left a comment

Choose a reason for hiding this comment

Uh oh!

bergzand commented Mar 20, 2020

Uh oh!

benpicco commented Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

benpicco commented Mar 22, 2020 •

edited

Loading