makefiles/buildtest: always execute 'buildtest' loop on host machine by cladmi · Pull Request #11857 · RIOT-OS/RIOT

cladmi · 2019-07-17T14:34:34Z

Contribution description

This remove executing buildtest for loop in docker.

When building completely in docker, 'buildtest' would hide issues when
the host toolchain would be used when doing make all directly.

A side effect is also that now BUILDTEST_MAKE_REDIRECT would work when
doing buildtest with docker.
This also now removes passing BOARDS as it was added and needed only
for 'buildtest', and should work without the specific handling. #11385

The side effect is that now one docker instance should be started per compilation but makes it more consistent with doing BUILD_IN_DOCKER=1 BOARD=board_name make for each board.

Testing procedure

I locally used DOCKER="sudo docker" but did not include it in the tests commands displayed below.

Normal buildtest uses

Running buildtest with BUILD_IN_DOCKER=1 still works.

BUILD_IN_DOCKER=1 make --no-print-directory -C examples/hello-world/ buildtest
Building for acd52832 ... success.
Building for airfy-beacon ... success.
...
Building for wsn430-v1_3b ... success.
Building for wsn430-v1_4 ... success.
Building for yunjia-nrf51822 ... success.
Building for z1 ... success.

BOARDS can still be limited:

BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro pic32-wifire' make --no-print-directory -C examples/hello-world/ buildtest
Building for native ... success.
Building for samr21-xpro ... success.
Building for pic32-wifire ... success.

When setting BUILDTEST_MAKE_REDIRECT= in the environment it will be taken into account

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/default buildtest

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/default buildtest
Building for native ... Launching build container using image "riot/riotbuild:latest".
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache   \
    -e 'BOARD=native' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/default/' \
    'riot/riotbuild:latest' make all
Building application "default" for "native" with MCU "native".

   text    data     bss     dec     hex filename
  90210    1104   72024  163338   27e0a /data/riotbuild/riotbase/examples/default/bin/native/default.elf
success.
Building for samr21-xpro ... Launching build container using image "riot/riotbuild:latest".
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache   \
    -e 'BOARD=samr21-xpro' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/default/' \
    'riot/riotbuild:latest' make all
Building application "default" for "samr21-xpro" with MCU "samd21".

   text    data     bss     dec     hex filename
  40856     508    6048   47412    b934 /data/riotbuild/riotbase/examples/default/bin/samr21-xpro/default.elf
success.

In master it was showing the docker command first to run buildtest now all is run with docker for each board.

Master:

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/hello-world/ buildtest

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/hello-world/ buildtest
Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/harter/work/git/worktree/riot_master/build
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/worktree/riot_master:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache  -v /home/harter/work/git/RIOT/.git:/home/harter/work/git/RIOT/.git \
    -e 'BOARDS=native samr21-xpro' \
    -w '/data/riotbuild/riotbase/examples/hello-world/' \
    'riot/riotbuild:latest' make buildtest  
Building for native ... success.
Building for samr21-xpro ... success.

Removes errors hiding

Use a PATH where you do not have arm-none-eabi-gcc

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin arm-none-eabi-gcc

Command 'arm-none-eabi-gcc' not found, but can be installed with:

sudo apt install gcc-arm-none-eabi

It now correctly fails with buildtest with BUILD_IN_DOCKER=1 as the host toolchain is used by that application.

BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest

BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest
make: Entering directory '/home/harter/work/git/RIOT/tests/riotboot'
Building for iotlab-m3 ... /bin/sh: 1: arm-none-eabi-gcc: not found
make[1]: *** [/home/harter/work/git/RIOT/tests/riotboot/bin/iotlab-m3/tests_riotboot-slot0.elf] Error 127
failed!
Building for samr21-xpro ... /bin/sh: 1: arm-none-eabi-gcc: not found
make[1]: *** [/home/harter/work/git/RIOT/tests/riotboot/bin/samr21-xpro/tests_riotboot-slot0.elf] Error 127
failed!
/home/harter/work/git/RIOT/makefiles/buildtests.inc.mk:6: recipe for target 'buildtest' failed
make: *** [buildtest] Error 1
make: Leaving directory '/home/harter/work/git/RIOT/tests/riotboot'

When it was hiding the error and correctly compiling in master

DOCKER="sudo docker" BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest

DOCKER="sudo docker" BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest
make: Entering directory '/home/harter/work/git/RIOT/tests/riotboot'
Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/harter/work/git/RIOT/build
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache   \
    -e 'BOARDS=iotlab-m3 samr21-xpro' \
    -w '/data/riotbuild/riotbase/tests/riotboot/' \
    'riot/riotbuild:latest' make buildtest
Building for iotlab-m3 ... «success.
Building for samr21-xpro ... success.
make: Leaving directory '/home/harter/work/git/RIOT/tests/riotboot'

and also hiding the error and correctly compiling in with buildtest-indocker.

BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest-indocker

``` BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest-indocker make: Entering directory '/home/harter/work/git/RIOT/tests/riotboot' Launching build container using image "riot/riotbuild:latest". mkdir -p /home/harter/work/git/RIOT/build sudo docker run --rm -t -u "$(id -u)" \ -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache \ -e 'BOARDS=iotlab-m3 samr21-xpro' \ -w '/data/riotbuild/riotbase/tests/riotboot/' \ 'riot/riotbuild:latest' make buildtest-indocker Building for iotlab-m3 ... success. Building for samr21-xpro ... success. make: Leaving directory '/home/harter/work/git/RIOT/tests/riotboot' ```

mcuboot is now working since https://github.com//pull/11083

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/mcuboot/ buildtest

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/mcuboot/ buildtest
make: Entering directory '/home/harter/work/git/RIOT/tests/mcuboot'
Building for nrf52dk ... /bin/sh: 1: arm-none-eabi-gcc: not found
make[1]: *** [mcuboot] Error 127
failed!
/home/harter/work/git/RIOT/makefiles/buildtests.inc.mk:6: recipe for target 'buildtest' failed
make: *** [buildtest] Error 1
make: Leaving directory '/home/harter/work/git/RIOT/tests/mcuboot'

When it was hiding the error and correctly compiling in master

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 make -C tests/mcuboot/ buildtest

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 make -C tests/mcuboot/ buildtest
make: Entering directory '/home/harter/work/git/worktree/riot_master/tests/mcuboot'
Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/harter/work/git/worktree/riot_master/build
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/worktree/riot_master:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache  -v /home/harter/work/git/RIOT/.git:/home/harter/work/git/RIOT/.git \
     \
    -w '/data/riotbuild/riotbase/tests/mcuboot/' \
    'riot/riotbuild:latest' make buildtest  
Building for nrf52dk ... success.
make: Leaving directory '/home/harter/work/git/worktree/riot_master/tests/mcuboot'

Issues/PRs references

Found that BUILD_IN_DOCKER=1 make buildtest was hiding expected errors during 2019.07-RC1 testing RIOT-OS/Release-Specs#128 (comment)

Fixes #11842

fjmolinas · 2019-08-08T09:36:51Z

Does build time increase in a significant way because docker is started per compilation?

cladmi · 2019-08-12T15:01:53Z

It does indeed increase I will do a measure on my machine.

However, if not changing it because of execution time, I would see it as a "disable test as it takes more time".
This shows issues that happen on a developer machine that murdock does not show as it is always inside the container and never with BUILD_IN_DOCKER=1.

cladmi · 2019-08-13T10:08:49Z

The time increase is indeed significant. It changes from 10 to 20 minutes for examples/hello-world.

Should it be split in two different targets? So buildtest would be the real test and a buildtest-no-local-toolchain-check something ?

fjmolinas · 2019-08-14T11:49:36Z

There is something I don't quite understand here, why should building in docker use the host tool chain in the first place?

cladmi · 2019-08-14T12:17:28Z

It MUST not use the host toolchain. That is why when testing building with docker on my test machine I do not have any local toolchain.
(except native gcc as it is required for flashing for kinetis, or compiling host 'edbg')

However, the BUILD_IN_DOCKER integration is based on an internal hook that calls make again inside a prepared docker container. For a normal compilation it is through this, that then is handled in makefiles/docker.inc.mk:

RIOT/Makefile.include

Lines 460 to 467 in 950b83e

    
           ifeq ($(BUILD_IN_DOCKER),1) 
        
           link: ..in-docker-container 
        
           else 
        
           ifeq (,$(RIOTNOLINK)) 
        
           link: ..compiler-check ..build-message $(ELFFILE) $(FLASHFILE) print-size 
        
           else 
        
           link: ..compiler-check ..build-message $(BASELIBS) 
        
           endif # RIOTNOLINK

But some applications, are currently handled in a way that the build system still executes targets outside of this handling:

tests/mcuboot tests/mcuboot: handle building in docker #11083
tests/riotboot, bootloader/riotboot bootloaders|tests/riotboot: broken with BUILD_IN_DOCKER and wrong flashfile #12003

Consequence on `buildtest`

With the current way buildtest is implemented with BUILD_IN_DOCKER, the compilation command:

RIOT/makefiles/buildtests.inc.mk

Lines 14 to 15 in 950b83e

    
           BOARD=$${board} RIOT_CI_BUILD=1 RIOT_VERSION_OVERRIDE=buildtest \ 
        
           	$(MAKE) clean all -j $(NPROC) $(BUILDTEST_MAKE_REDIRECT); \

is executed from inside the container, so without BUILD_IN_DOCKER=1.
Because of this, it never uses the handling in Makefile.include directly on your machine.
And this hides the issues you would have when doing BUILD_IN_DOCKER=1 make all and that I would like to have visible.

fjmolinas · 2019-08-14T12:34:04Z

@cladmi Ok now I understand the issue correctly.

So buildtest would be the real test and a buildtest-no-local-toolchain-check something ?

+1 for this approach, all tough I would rename it to buildtests-indocker or something like that. Would murdock need to be changed to avoid the x2 in build time? I think it's good enough if during release testing we use just build-test and buildtests-indocker normally. What do you think?

cladmi · 2019-08-14T12:47:13Z

Murdock is not using the BUILD_IN_DOCKER=1 at all, and not buildtest either. So no impact there.

It is only during release, or developers on their machine that would run buildtest.

Good for buildtest-indocker, I will update it.

cladmi · 2019-08-14T13:03:36Z

PR needs squashing.

cladmi · 2019-08-14T13:22:23Z

The testing prodecures from the first post are still valid and get the same output.

The new buildtest-indocker gives the same outpus as buildtest before with BUILD_IN_DOCKER=1:

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 DOCKER="sudo docker" BOARDS="native samr21-xpro" make --no-print-directory -C examples/default/ buildtest-indocker

BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 DOCKER="sudo docker" BOARDS="native samr21-xpro" make --no-print-directory -C examples/default/ buildtest-indocker
Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/harter/work/git/RIOT/build
sudo docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/harter/work/git/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -v /home/harter/.gitcache:/data/riotbuild/gitcache -e GIT_CACHE_DIR=/data/riotbuild/gitcache   \
    -e 'BOARDS=native samr21-xpro' \
    -w '/data/riotbuild/riotbase/examples/default/' \
    'riot/riotbuild:latest' make buildtest-indocker
Building for native ... success.
Building for samr21-xpro ... success.

And when executed without BUILD_IN_DOCKER=1 it displays an error:

make --no-print-directory -C examples/default/ buildtest-indocker
/home/harter/work/git/RIOT/makefiles/buildtests.inc.mk:35: *** buildtest-indocker must be run with `BUILD_IN_DOCKER=1`.  Stop.

cladmi · 2019-08-14T13:34:51Z

I rebased now that #11083 was merged and updated the testing procedure in the PR description to use tests/riotboot.

fjmolinas · 2019-08-14T14:42:23Z

Verified test procedure.

master: BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/default buildtest

Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/francisco/workspace/RIOT/build
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARDS=native samr21-xpro' \
    -w '/data/riotbuild/riotbase/examples/default/' \
    'riot/riotbuild:latest' make buildtest  
Building for native ... success.
Building for samr21-xpro ... success.

PR: BUILDTEST_MAKE_REDIRECT= BUILD_IN_DOCKER=1 BOARDS='native samr21-xpro' make --no-print-directory -C examples/default buildtest

Building for native ... Launching build container using image "riot/riotbuild:latest".
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARD=native' -e 'BOARDS=native samr21-xpro' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/hello-world/' \
    'riot/riotbuild:latest' make all  
Building application "hello-world" for "native" with MCU "native".

   text	   data	    bss	    dec	    hex	filename
  22918	    568	  47652	  71138	  115e2	/data/riotbuild/riotbase/examples/hello-world/bin/native/hello-world.elf
success.
Building for samr21-xpro ... Launching build container using image "riot/riotbuild:latest".
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbaseBuilding for native ... Launching build container using image "riot/riotbuild:latest".
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARD=native' -e 'BOARDS=native samr21-xpro' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/hello-world/' \
    'riot/riotbuild:latest' make all  
Building application "hello-world" for "native" with MCU "native".

   text	   data	    bss	    dec	    hex	filename
  22918	    568	  47652	  71138	  115e2	/data/riotbuild/riotbase/examples/hello-world/bin/native/hello-world.elf
success.
Building for samr21-xpro ... Launching build container using image "riot/riotbuild:latest".
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARD=samr21-xpro' -e 'BOARDS=native samr21-xpro' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/hello-world/' \
    'riot/riotbuild:latest' make all  
Building application "hello-world" for "samr21-xpro" with MCU "samd21".

   text	   data	    bss	    dec	    hex	filename
   8216	    120	   2548	  10884	   2a84	/data/riotbuild/riotbase/examples/hello-world/bin/samr21-xpro/hello-world.elf' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARD=samr21-xpro' -e 'BOARDS=native samr21-xpro' -e 'RIOT_CI_BUILD=1' \
    -w '/data/riotbuild/riotbase/examples/hello-world/' \
    'riot/riotbuild:latest' make all  
Building application "hello-world" for "samr21-xpro" with MCU "samd21".

   text	   data	    bss	    dec	    hex	filename
   8216	    120	   2548	  10884	   2a84	/data/riotbuild/riotbase/examples/hello-world/bin/samr21-xpro/hello-world.elf

master:BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest

Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/francisco/workspace/RIOT/build
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARDS=iotlab-m3 samr21-xpro' \
    -w '/data/riotbuild/riotbase/tests/riotboot/' \
    'riot/riotbuild:latest' make buildtest  
Building for iotlab-m3 ... success.
Building for samr21-xpro ... success.
make: Leaving directory '/home/francisco/workspace/RIOT/tests/riotboot'

PR:BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest

make: Entering directory '/home/francisco/workspace/RIOT/tests/riotboot'
Building for iotlab-m3 ... /bin/sh: 1: arm-none-eabi-gcc: not found
make[1]: *** [/home/francisco/workspace/RIOT/tests/riotboot/bin/iotlab-m3/tests_riotboot-slot0.elf] Error 127
failed!
Building for samr21-xpro ... /bin/sh: 1: arm-none-eabi-gcc: not found
make[1]: *** [/home/francisco/workspace/RIOT/tests/riotboot/bin/samr21-xpro/tests_riotboot-slot0.elf] Error 127
failed!
/home/francisco/workspace/RIOT/makefiles/buildtests.inc.mk:6: recipe for target 'buildtest' failed
make: *** [buildtest] Error 1
make: Leaving directory '/home/francisco/workspace/RIOT/tests/riotboot'

PR:BOARDS="iotlab-m3 samr21-xpro" PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BUILD_IN_DOCKER=1 BUILDTEST_MAKE_REDIRECT='>/dev/null' make -C tests/riotboot buildtest-indocker

make: Entering directory '/home/francisco/workspace/RIOT/tests/riotboot'
Launching build container using image "riot/riotbuild:latest".
mkdir -p /home/francisco/workspace/RIOT/build
docker run --rm -t -u "$(id -u)" \
    -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'     \
    -e 'BOARDS=iotlab-m3 samr21-xpro' \
    -w '/data/riotbuild/riotbase/tests/riotboot/' \
    'riot/riotbuild:latest' make buildtest-indocker  
Building for iotlab-m3 ... success.
Building for samr21-xpro ... success.
make: Leaving directory '/home/francisco/workspace/RIOT/tests/riotboot'

fjmolinas

I agree with the changes this PR introduces, it exposes issues when building in docker while allowing to keep the same behavior as before when using the buildtest-target. I was able to reproduce the test procedure correctly for the case that exposed the issue in #11842. ACK, please squash.

Add a 'buildtest-indocker' that forces executing 'buildtest' for loop completely inside the container. It prevents starting one container per compilation wich is slower but it could hide errors where the host toolchain would be used It is currently equivalent to `buildtest` but will change when the `buidtest` handling will be move outside of `BUILD_IN_DOCKER`. Display an error when executed without BUILD_IN_DOCKER=1.

This remove executing buildtest `for` loop in docker. When building completely in docker, 'buildtest' would hide issues when the host toolchain would be used when doing `make all` directly. It has the consequence that it now starts a container for each compilation which is slower. The previous behavior can be reproduced by using BUILD_IN_DOCKER=1 make buildtest-indocker A side effect is also that now `BUILDTEST_MAKE_REDIRECT` would work when doing `buildtest` with docker.

cladmi · 2019-08-14T15:03:39Z

I squashed the commits and commit message.

fjmolinas · 2019-08-14T15:53:15Z

All green, GO!

cladmi · 2019-08-14T16:04:55Z

Thank you for the review! :)

cladmi mentioned this pull request Jul 17, 2019

Release 2019.07 - RC1 RIOT-OS/Release-Specs#128

Closed

77 tasks

MrKevinWeiss requested a review from kaspar030 July 17, 2019 15:01

MrKevinWeiss added Area: build system Area: Build system Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) labels Jul 17, 2019

cladmi mentioned this pull request Jul 17, 2019

buildtest with BUILD_IN_DOCKER hides host toolchain errors #11842

Closed

cladmi force-pushed the pr/make/buildtest/docker branch from cad1233 to 75ea03c Compare August 14, 2019 13:33

fjmolinas approved these changes Aug 14, 2019

View reviewed changes

cladmi added 2 commits August 14, 2019 17:02

cladmi force-pushed the pr/make/buildtest/docker branch from 75ea03c to 7d10da8 Compare August 14, 2019 15:03

fjmolinas added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: needs squashing Commits in this PR need to be squashed; If set, CI systems will mark this PR as unmergable labels Aug 14, 2019

fjmolinas merged commit 3e75383 into RIOT-OS:master Aug 14, 2019

cladmi deleted the pr/make/buildtest/docker branch August 14, 2019 16:07

kb2ma added this to the Release 2019.10 milestone Sep 16, 2019

Conversation

cladmi commented Jul 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Testing procedure

Normal buildtest uses

Removes errors hiding

Issues/PRs references

Uh oh!

fjmolinas commented Aug 8, 2019

Uh oh!

cladmi commented Aug 12, 2019

Uh oh!

cladmi commented Aug 13, 2019

Uh oh!

fjmolinas commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Consequence on buildtest

Uh oh!

fjmolinas commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

fjmolinas commented Aug 14, 2019

Uh oh!

fjmolinas left a comment

Choose a reason for hiding this comment

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

fjmolinas commented Aug 14, 2019

Uh oh!

cladmi commented Aug 14, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cladmi commented Jul 17, 2019 •

edited

Loading

Consequence on `buildtest`