Skip to content

tools: introduce shared tooling for using BOSSA#7068

Merged
aabadie merged 6 commits intoRIOT-OS:masterfrom
haukepetersen:add_tool_bossa
May 23, 2017
Merged

tools: introduce shared tooling for using BOSSA#7068
aabadie merged 6 commits intoRIOT-OS:masterfrom
haukepetersen:add_tool_bossa

Conversation

@haukepetersen
Copy link
Copy Markdown
Contributor

Some boards (arduino-due, udoo, arduino-mkr1000) shipped with some binaries for flashing, a tool called BOSSA (bossac). This is not only ugly, but does also bring problems with different host systems (32 vs 64 bit, different system compiler version...). The tool itself is open source and hosted on github, so there is actually no problem in downloading and building it ourselves.

So what happening now is:

  • you say make flash for your board (e.g. the arduino-due). Per default, RIOT tries to use it's own bossac binary
  • if the binary does not exist, the makesystem will automatically check out the BOSSA source from github and will build the bossac binary from it
  • the board will be flashed using the (potentially newly created) binary tool

This PR brings the following improvements:

  • a Makefile that downlaods and builds bossac from source
  • a shared Makefile makefiles/tools/bossa.inc.mk that can be used from different boards
  • a shared shell script dist/tools/bossa/bossa.sh that takes care of actually flashing a board
  • made all boards using BOSSA to use this new infrastructure and deleted the shipped binaries for these boards...

Open issue:

  • the output of the flash script could and the BOSSA makefile could probably be beautified, but I would like to leave this for a follow up PR

Needs proper testing!

@haukepetersen haukepetersen added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: build system Area: Build system Area: tools Area: Supplementary tools labels May 17, 2017
@haukepetersen haukepetersen added this to the Release 2017.07 milestone May 17, 2017
@jnohlgard
Copy link
Copy Markdown
Member

I approve the idea and the plan in the PR description, but I'm not really able to a code review presently @aabadie or @kaspar030 will have to do it.

@haukepetersen haukepetersen removed the request for review from jnohlgard May 17, 2017 10:49
@haukepetersen
Copy link
Copy Markdown
Contributor Author

thanks for the heads up!

Copy link
Copy Markdown
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a typo ! Otherwise it looks good codewise.

elif [ `uname` = "Darwin" ]; then
STTY_FLAG=-f
else
echo "error: your host system is not support by the bossa.sh script"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supported

# define path to bossac binary
export BOSSAC ?= $(RIOTBASE)/dist/tools/bossa/bossac

# if we go with the default (BOSSA shipped with RIOT), we download and build
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The autot build of bossa doesn't work for me. I have to build it first in dist:

$ make BOARD=arduino-mkr1000 -C examples/default flash
make: Entering directory '/home/aabadie/softs/src/riot/RIOT/examples/default'
Building application "default" for "arduino-mkr1000" with MCU "samd21".
[...]
env -u CXX make -C /home/aabadie/softs/src/riot/RIOT/dist/tools/bossa
mkdir -p /home/aabadie/softs/src/riot/RIOT/dist/tools/bossa/src
git clone "https://github.com/shumatech/BOSSA" "/home/aabadie/softs/src/riot/RIOT/dist/tools/bossa/src"
Cloning into '/home/aabadie/softs/src/riot/RIOT/dist/tools/bossa/src'...
[...]
COMMON_CXXFLAGS="-std=c++11" make -C /home/aabadie/softs/src/riot/RIOT/dist/tools/bossa/src strip-bossac
CPP APPLET src/WordCopyArm.cpp
CPP COMMON src/Samba.cpp
CPP COMMON src/Flash.cpp
[...]
LD /home/aabadie/softs/src/riot/RIOT/examples/default/bin/arduino-mkr1000/bossac
STRIP /home/aabadie/softs/src/riot/RIOT/examples/default/bin/arduino-mkr1000/bossac
mv: cannot stat '/home/aabadie/softs/src/riot/RIOT/dist/tools/bossa/src/bin/bossac': No such file or directory

As you can see, the BOSSA Makefile creates the executable in the application directory.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, will check where this comes from

@haukepetersen
Copy link
Copy Markdown
Contributor Author

fixed the issue above and found one more that I fixed. In summery:

  • paths for building bossa should now always be correct, independently from where the makefile is called
  • my first try did override the default target, so a plain make always lead to make flash instead of make all as one would expect. This is solved by moving the .../bossac target into a separate makefile (tools/targets.inc.mk), that is included in the main Makefile after the default targets have been defined.

@haukepetersen
Copy link
Copy Markdown
Contributor Author

@aabadie would you mind to give this another test with your boards? Thanks!

fi

# now we flash the binary onto the target board
${BOSSAC} --port="${PORT}" ${BOSSAC_FLAGS} "${HEXFILE}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the script to not use environment but cmdline for options (as far as easily possible).

Setting

FFLAGS:="-i -e -w -v -b -R --PORT=$(PORT) $(HEXFILE)"

in makefiles/tools/bossac.mk and using

${BOSSAC} $*

here keeps the configuration whithin the makefiles, and more consistent for different flashers.

Ideally you'd also create a phony target for stty, depend on that for flashing, and drop the shell script altogether.

@haukepetersen
Copy link
Copy Markdown
Contributor Author

done: removed the shell script and moved everything into bossa.inc.mk. Solved the stty call by introducing a global preflash target that is only called on demand (i.e. by adding it to FLASHDEPS).

@aabadie
Copy link
Copy Markdown
Contributor

aabadie commented May 18, 2017

Gave another try but

$ make BOARD=arduino-mkr1000 -C examples/default flash
or
$ cd examples/default && make BOARD=arduino-mkr1000 flash

still fails for the same reason.

I'm not expert with Makefile so I can't really help. Maybe adding an error message if bossac is not present in either PATH or dist/tools/bossa would be enough for the moment. what do you think ?

@haukepetersen
Copy link
Copy Markdown
Contributor Author

ups, seems like my last change broke something again. Will fix it

@haukepetersen
Copy link
Copy Markdown
Contributor Author

its an issue I had earlier today, but though it was solved: the env -u BINDIR ... statement in targets.inc.mk doesn't seem to have any effect...

@haukepetersen
Copy link
Copy Markdown
Contributor Author

just gave it another test with a freshly booted host system: it works as expected for me running

BOARD=arduino-due make -C examples/default flash 

from RIOTBASE dir:

hauke@prag ~/dev/riot/RIOT $ BOARD=arduino-due make -C examples/default flash
make: Entering directory `/home/hauke/dev/riot/RIOT/examples/default'
Building application "default" for "arduino-due" with MCU "sam3".

"make" -C /home/hauke/dev/riot/RIOT/boards/arduino-due
"make" -C /home/hauke/dev/riot/RIOT/core
"make" -C /home/hauke/dev/riot/RIOT/cpu/sam3
"make" -C /home/hauke/dev/riot/RIOT/cpu/cortexm_common
"make" -C /home/hauke/dev/riot/RIOT/cpu/cortexm_common/periph
"make" -C /home/hauke/dev/riot/RIOT/cpu/sam3/periph
"make" -C /home/hauke/dev/riot/RIOT/drivers
"make" -C /home/hauke/dev/riot/RIOT/drivers/periph_common
"make" -C /home/hauke/dev/riot/RIOT/drivers/saul
"make" -C /home/hauke/dev/riot/RIOT/sys
"make" -C /home/hauke/dev/riot/RIOT/sys/auto_init
"make" -C /home/hauke/dev/riot/RIOT/sys/auto_init/saul
"make" -C /home/hauke/dev/riot/RIOT/sys/fmt
"make" -C /home/hauke/dev/riot/RIOT/sys/isrpipe
"make" -C /home/hauke/dev/riot/RIOT/sys/newlib
"make" -C /home/hauke/dev/riot/RIOT/sys/phydat
"make" -C /home/hauke/dev/riot/RIOT/sys/ps
"make" -C /home/hauke/dev/riot/RIOT/sys/saul_reg
"make" -C /home/hauke/dev/riot/RIOT/sys/shell
"make" -C /home/hauke/dev/riot/RIOT/sys/shell/commands
"make" -C /home/hauke/dev/riot/RIOT/sys/tsrb
"make" -C /home/hauke/dev/riot/RIOT/sys/uart_stdio
   text	   data	    bss	    dec	    hex	filename
  13564	    564	   2712	  16840	   41c8	/home/hauke/dev/riot/RIOT/examples/default/bin/arduino-due/default.elf
stty -F /dev/ttyACM0 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
[INFO] bossac binary not found - building it from source now
[INFO] checking out BOSSA source code from github now
Cloning into '/home/hauke/dev/riot/RIOT/dist/tools/bossa/src'...
remote: Counting objects: 1306, done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 1306 (delta 16), reused 18 (delta 6), pack-reused 1262
Receiving objects: 100% (1306/1306), 1.06 MiB | 0 bytes/s, done.
Resolving deltas: 100% (972/972), done.
Checking connectivity... done.
[INFO] compiling bossac from source now
CPP APPLET src/WordCopyArm.cpp
CPP COMMON src/Samba.cpp
CPP COMMON src/Flash.cpp
CPP COMMON src/NvmFlash.cpp
CPP COMMON src/EfcFlash.cpp
CPP COMMON src/EefcFlash.cpp
CPP COMMON src/FlashFactory.cpp
CPP COMMON src/Applet.cpp
CPP COMMON src/WordCopyApplet.cpp
CPP COMMON src/Flasher.cpp
CPP COMMON src/PosixSerialPort.cpp
CPP COMMON src/LinuxPortFactory.cpp
CPP BOSSAC src/bossac.cpp
CPP BOSSAC src/CmdOpts.cpp
LD bin/bossac
STRIP bin/bossac
[INFO] bossac binary successfully build!
/home/hauke/dev/riot/RIOT/dist/tools/bossa/bossac -p /dev/ttyACM0 -i -i -w -v -b -R /home/hauke/dev/riot/RIOT/examples/default/bin/arduino-due/default.bin
Device       : ATSAM3X8
Chip ID      : 285e0a60
Version      : v1.1 Dec 15 2010 19:25:04
Address      : 524288
Pages        : 2048
Page Size    : 256 bytes
Total Size   : 512KB
Planes       : 2
Lock Regions : 32
Locked       : 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
Security     : false
Boot Flash   : false
Write 14128 bytes to flash (56 pages)
[==============================] 100% (56/56 pages)
Done in 2.736 seconds
Verify 14128 bytes of flash
[==============================] 100% (56/56 pages)
Verify successful
Done in 2.527 seconds
Set boot flash true
CPU reset.
make: Leaving directory `/home/hauke/dev/riot/RIOT/examples/default'

@aabadie
Copy link
Copy Markdown
Contributor

aabadie commented May 19, 2017

Tested this one again following the previous comment where there is a subtle difference with the command I tried:
you issued:

BOARD=arduino-due make -C examples/default flash 

I did:

make BOARD=arduino-due -C examples/default flash 

The first one works and the second doesn't. So not passing BOARD as an environment variable messes up the destination directory of bossac.

@@ -0,0 +1,25 @@
NAME = bossa
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use pkg.mk as done here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait a minute, this PR was first :-). But I will adjust this, makes sense.

@haukepetersen
Copy link
Copy Markdown
Contributor Author

@aabadie: I see, so I more debugging is needed. I wonder why this subtle difference is screwing thing up...

@haukepetersen
Copy link
Copy Markdown
Contributor Author

@kaspar030, @aabadie now using pkg.mk and repaired the env var problem, at least on my host it now also builds fine with make BOARD=arduino-due -C examples/default flash :

hauke@prag ~/dev/riot/RIOT $ make BOARD=arduino-due -C examples/default flash 
make: Entering directory `/home/hauke/dev/riot/RIOT/examples/default'
Building application "default" for "arduino-due" with MCU "sam3".

"make" -C /home/hauke/dev/riot/RIOT/boards/arduino-due
"make" -C /home/hauke/dev/riot/RIOT/core
"make" -C /home/hauke/dev/riot/RIOT/cpu/sam3
"make" -C /home/hauke/dev/riot/RIOT/cpu/cortexm_common
"make" -C /home/hauke/dev/riot/RIOT/cpu/cortexm_common/periph
"make" -C /home/hauke/dev/riot/RIOT/cpu/sam3/periph
"make" -C /home/hauke/dev/riot/RIOT/drivers
"make" -C /home/hauke/dev/riot/RIOT/drivers/periph_common
"make" -C /home/hauke/dev/riot/RIOT/drivers/saul
"make" -C /home/hauke/dev/riot/RIOT/sys
"make" -C /home/hauke/dev/riot/RIOT/sys/auto_init
"make" -C /home/hauke/dev/riot/RIOT/sys/auto_init/saul
"make" -C /home/hauke/dev/riot/RIOT/sys/fmt
"make" -C /home/hauke/dev/riot/RIOT/sys/isrpipe
"make" -C /home/hauke/dev/riot/RIOT/sys/newlib
"make" -C /home/hauke/dev/riot/RIOT/sys/phydat
"make" -C /home/hauke/dev/riot/RIOT/sys/ps
"make" -C /home/hauke/dev/riot/RIOT/sys/saul_reg
"make" -C /home/hauke/dev/riot/RIOT/sys/shell
"make" -C /home/hauke/dev/riot/RIOT/sys/shell/commands
"make" -C /home/hauke/dev/riot/RIOT/sys/tsrb
"make" -C /home/hauke/dev/riot/RIOT/sys/uart_stdio
   text	   data	    bss	    dec	    hex	filename
  13564	    564	   2712	  16840	   41c8	/home/hauke/dev/riot/RIOT/examples/default/bin/arduino-due/default.elf
stty -F /dev/ttyACM0 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
[INFO] bossac binary not found - building it from source
rm -Rf /home/hauke/dev/riot/RIOT/dist/tools/bossa/bin
mkdir -p /home/hauke/dev/riot/RIOT/dist/tools/bossa/bin
/home/hauke/dev/riot/RIOT/dist/tools/git/git-cache clone "https://github.com/shumatech/BOSSA" "26154375695f345491bba158d57177aa231d6765" "/home/hauke/dev/riot/RIOT/dist/tools/bossa/bin"
Cloning into '/home/hauke/dev/riot/RIOT/dist/tools/bossa/bin'...
remote: Counting objects: 1381, done.
remote: Compressing objects: 100% (77/77), done.
remote: Total 1381 (delta 66), reused 81 (delta 39), pack-reused 1265
Receiving objects: 100% (1381/1381), 1.10 MiB | 0 bytes/s, done.
Resolving deltas: 100% (1024/1024), done.
Checking connectivity... done.
HEAD is now at 2615437... Fixed problem with lock regions on SAM4E.  Some general cleanup.
touch /home/hauke/dev/riot/RIOT/dist/tools/bossa/bin/.git-downloaded
[INFO] compiling bossac from source now
CPP APPLET src/WordCopyArm.cpp
CPP COMMON src/Samba.cpp
CPP COMMON src/Flash.cpp
CPP COMMON src/NvmFlash.cpp
CPP COMMON src/EfcFlash.cpp
CPP COMMON src/EefcFlash.cpp
CPP COMMON src/FlashFactory.cpp
CPP COMMON src/Applet.cpp
CPP COMMON src/WordCopyApplet.cpp
CPP COMMON src/Flasher.cpp
CPP COMMON src/PosixSerialPort.cpp
CPP COMMON src/LinuxPortFactory.cpp
CPP BOSSAC src/bossac.cpp
CPP BOSSAC src/CmdOpts.cpp
LD /home/hauke/dev/riot/RIOT/dist/tools/bossa/bin/bossac
STRIP /home/hauke/dev/riot/RIOT/dist/tools/bossa/bin/bossac
[INFO] bossac binary successfully build!
/home/hauke/dev/riot/RIOT/dist/tools/bossa/bossac -p /dev/ttyACM0 -i -i -w -v -b -R /home/hauke/dev/riot/RIOT/examples/default/bin/arduino-due/default.bin
Device       : ATSAM3X8
Chip ID      : 285e0a60
Version      : v1.1 Dec 15 2010 19:25:04
Address      : 524288
Pages        : 2048
Page Size    : 256 bytes
Total Size   : 512KB
Planes       : 2
Lock Regions : 32
Locked       : none
Security     : false
Boot Flash   : false
Write 14128 bytes to flash (56 pages)
[==============================] 100% (56/56 pages)
Done in 2.736 seconds
Verify 14128 bytes of flash
[==============================] 100% (56/56 pages)
Verify successful
Done in 2.527 seconds
Set boot flash true
CPU reset.
make: Leaving directory `/home/hauke/dev/riot/RIOT/examples/default'

Copy link
Copy Markdown
Contributor

@kaspar030 kaspar030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nitpick regarding port name change.
Tested successfully on arduino due.
Don't have the other boards.

export OFLAGS = -O binary
export FFLAGS = -R -e -w -v -b bin/$(BOARD)/$(APPLICATION).hex
# define the default port depending on host OS
PORT_LINUX ?= /dev/ttyACM0
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default port name changed, is that intended?

@kaspar030
Copy link
Copy Markdown
Contributor

repaired the env var problem

what was it?

@echo "[INFO] compiling bossac from source now"
@env -u CXX COMMON_CXXFLAGS="-std=c++11" $(MAKE) BINDIR=$(PKG_BUILDDIR) -C $(PKG_BUILDDIR) strip-bossac
@mv $(PKG_BUILDDIR)/bossac $(CURDIR)/bossac

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to add a clean target ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be implicit by using pkg.mk. try "make distclean"

@haukepetersen
Copy link
Copy Markdown
Contributor Author

what was it?

Don't know if I really can explain what it was exactly, but when building with make BOARD=arduino-due -C examples/default flash it seems that the BINDIR variable was set in a way, so that calling the BOSSA Makefile with env -u BINDIR make ... did not reset the BINDIR var, but left it to point to the RIOT bindir, which confused the build process of BOSSA and prevented it from creating the bossac binary in the correct place. Now calling the BOSSA Makfile with make BINDIR=xxx ... does actually override that var, so with this it works as expected...

@haukepetersen
Copy link
Copy Markdown
Contributor Author

should be implicit by using pkg.mk. try "make distclean"

Hm, thought so, too. But just tried it out, and make distclean does not clean the BOSSA files...

@kaspar030
Copy link
Copy Markdown
Contributor

Hm, thought so, too. But just tried it out, and make distclean does not clean the BOSSA files...

RIOTBASE=$(pwd) make -C dist/tools/bossa/ distclean

... cleans bin/. "bossac" would need an extra rule in "dist/tools/bossa/":

distclean::
    @rm -f $(CURDIR)/bossac

... and calling distclean for host tools should go into another PR.

@kaspar030
Copy link
Copy Markdown
Contributor

would need an extra rule

I pushed that into your branch.

@env -u CXX COMMON_CXXFLAGS="-std=c++11" $(MAKE) BINDIR=$(PKG_BUILDDIR) -C $(PKG_BUILDDIR) strip-bossac
@mv $(PKG_BUILDDIR)/bossac $(CURDIR)/bossac

distclean::
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why ::. I thought : is enough.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I'll have to read the Makefile manual in detail one day...

@kaspar030 kaspar030 added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label May 22, 2017
@kaspar030
Copy link
Copy Markdown
Contributor

@aabadie PR's all yours. ;)

Copy link
Copy Markdown
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@aabadie aabadie merged commit 0b918db into RIOT-OS:master May 23, 2017
@haukepetersen haukepetersen deleted the add_tool_bossa branch May 23, 2017 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: build system Area: Build system Area: tools Area: Supplementary tools CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants