cpu/esp: cleanup of ESP SDK log outputs by gschorcht · Pull Request #12998 · RIOT-OS/RIOT

gschorcht · 2019-12-20T16:55:26Z

Contribution description

The ESP SDK libraries generate a lot of log outputs at the default log level, which are usually not of interest. Especially, the WiFi SDK libraries that are used by esp_now and esp_wifi produce a lot of such output about driver internals, for example

add if0
add if1
bcn 100
main(): This is RIOT! (Version: 2020.01-devel-1500-g31330-cpu/esp ...)
RIOT network stack example application
All up, running the shell now
> scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 15
cnt 

connected with SSID, channel 8
WiFi connected to ssid SSID, channel 8

This PR implements a set of library-specific printf functions that intercept the log output from the SDK libraries and redirect them to the RIOT log macros. This allows to control the log output of the ESP SDK libraries according to the defined log level.

The PR contains also some small cleanups of log outputs in initialization functions.

Note: This PR requires a modified SDK version for ESP8266 boards to use the changes. The modified SDK version is available at GitHub.

Testing procedure

Test any ESP32 board with:

USEMODULE='esp_wifi' CFLAGS='-DESP_WIFI_SSID=\"pass\" -DESP_WIFI_PASS=\"ssid\"' \
make BOARD=esp32-wroom-32 -C examples/gnrc_networking flash term

LOG_LEVEL=4 USEMODULE='esp_wifi' CFLAGS='-DESP_WIFI_SSID=\"pass\" -DESP_WIFI_PASS=\"ssid\"' \
make BOARD=esp32-wroom-32 -C examples/gnrc_networking flash term

With LOG_LEVEL=4, you should see the log output as before. At default log level you should only see

main(): This is RIOT! (Version: 2020.01-devel-1500-g31330-cpu/esp ...)
RIOT network stack example application
All up, running the shell now
> WiFi connected to ssid SSID, channel 8

The test of the PR with an ESP8266 board requires a modified SDK Version. Without the modified version of the modified SDK Version, ESP8266 should be compilable. The modified version of the SDK can be installed with:

cd /path/to
git clone https://github.com/gschorcht/RIOT-Xtensa-ESP8266-RTOS-SDK.git
git checkout release/v3.1-for-riot-os-v2
export ESP8266_RTOS_SDK_DIR=/path/to/RIOT-Xtensa-ESP8266-RTOS-SDK

Once the modified SDK version is installed, you can use:

USEMODULE='esp_wifi' CFLAGS='-DESP_WIFI_SSID=\"pass\" -DESP_WIFI_PASS=\"ssid\"' \
make BOARD=esp8266-esp-12x -C examples/gnrc_networking flash term

LOG_LEVEL=4 USEMODULE='esp_wifi' CFLAGS='-DESP_WIFI_SSID=\"pass\" -DESP_WIFI_PASS=\"ssid\"' \
make BOARD=esp8266-esp-12x -C examples/gnrc_networking flash term

Issues/PRs references

This PR is prerequisite for PR RIOT-OS/riotdocker#95

LOG_TAG macro is required for LOG_VERBOSE level in ESP SDK log output handling.

The log level for normal information should be LOG_DEBUG.

gschorcht · 2019-12-20T16:57:54Z

@benpicco This PR could be interesting for you. For example, it solves the problem with bunches scandone messages with esp_now

To control the log level and the format of the log output of SDK libraries, a bunch of library-specific printf functions are realized which map the log output from SDK libraries to RIOT's log macros.

gschorcht · 2020-01-07T00:40:45Z

@benpicco Do you think that there is a chance to get this PR merged before the soft freeze of release 2020.01. The reason for my question is that a modified tool chain is required for this PR to take affect on the ESP8266, see RIOT-OS/riotdocker#95 and we have to merge this PR before we can merge RIOT-OS/riotdocker#95.

If this PR will not become part of release 2020.01, we would have to keep RIOT-OS/riotdocker#95 on hold until this PR is released with 2020.04. Otherwise ESP8266 would not be compilable anymore with the current release.

It would be OK for me, if you think it's too short notice. It was just a question.

benpicco · 2020-01-11T16:25:46Z

Sorry, I've been sick the last couple days.
This looks rather straightforward though. And I enjoy the silence :)

Btw.: since the SDK version tends to move in tandem with the RIOT code, have you thought of moving it to pkg/ so it will automatically be the right version?

gschorcht · 2020-01-12T02:54:15Z

Btw.: since the SDK version tends to move in tandem with the RIOT code, have you thought of moving it to pkg/ so it will automatically be the right version?

Yeah, but it doesn't seem like a reasonable way. The ESP8266 SDK contains a lot of binary blobs and has a size of about 109 MByte. Using it as a package would pull the SDK for each compiled application. It would slow down the compilation a lot and would take a lot of disk space, for example if you compile all applications.

gschorcht · 2020-01-12T03:35:40Z

The SDK contains a lot of binary blobs and has a size of about 109 MByte.

OK, if I prepare a lightweight ESP8266 SDK by removing everything but the required header files and binary libraries, the SDK would only have 5.9 MB. But 5.9 MB can still be a lot if you have a very slow internet connection and it has to be pulled on every compilation.

gschorcht · 2020-01-14T17:06:42Z

@benpicco All lights are green.

gschorcht · 2020-01-15T08:25:55Z

Today is hard feature freeze. Can wie merge it?

benpicco · 2020-01-15T14:07:57Z

If you want it in, we should merge it.

fjmolinas · 2020-01-15T14:24:20Z

Note: This PR requires a modified SDK version for ESP8266 boards to use the changes. The modified SDK version is available at GitHub.

Does this work in master? Can I still build with docker? @gschorcht @benpicco

Today is hard feature freeze. Can wie merge it?

BTW if there was a reason to question this, AFAIK I should have been consulted.

fjmolinas · 2020-01-15T14:33:27Z

~~Does this work in master? Can I still build with docker? @gschorcht @benpicco~~

Well If murdock passes it builds, but I would like to know why you though there was need of holding back.

gschorcht · 2020-01-15T14:55:49Z

Does this work in master? Can I still build with docker? @gschorcht @benpicco

The PR has only added some new functions that are required by the ESP8266 toolchain update in PR RIOT-OS/riotdocker#95. They are not yet used but will take effect as soon as the toolchain in the docker is updated. That is, the toolchain update depends on this PR. That's why this PR had to merge first even though it doesn't take effect on ESP8266 at the moment. I just wanted to have it in release 2020.01 so that we are able to do the toolchain update in next few month.

fjmolinas · 2020-01-15T14:57:59Z

The PR has only added some new functions that are required by the ESP8266 toolchain update in PR RIOT-OS/riotdocker#95. They are not yet used but will take effect as soon as the toolchain in the docker is updated. That is, the toolchain update depends on this PR. That's why this PR had to merge first even though it doesn't take effect on ESP8266 at the moment. I just wanted to have it in release 2020.01 so that we are able to do the toolchain update in next few month.

Ok thanks for the clarification, sorry for stressing for nothing.

gschorcht · 2020-01-16T13:15:44Z

@benpicco Thanks for reviewing and merging this PR.

gschorcht added 3 commits December 20, 2019 16:25

cpu/esp32: unify esp_wifi log output

40b878e

cpu/esp_common: LOG_TAG macro for DEBUG_ALL

7bf5bba

LOG_TAG macro is required for LOG_VERBOSE level in ESP SDK log output handling.

cpu/esp*: cleanup the log level for init functions

649bce4

The log level for normal information should be LOG_DEBUG.

gschorcht added 2 commits December 20, 2019 17:58

cpu/esp_common: functions for SDK log outputs

1899c25

To control the log level and the format of the log output of SDK libraries, a bunch of library-specific printf functions are realized which map the log output from SDK libraries to RIOT's log macros.

cpu/esp32: make the level for wifi log outputs settable

53139c0

gschorcht force-pushed the cpu/esp/common_sdk_log_output branch from 34ce3af to 53139c0 Compare December 20, 2019 16:59

gschorcht mentioned this pull request Dec 22, 2019

esp8266: replacement of SDK library printf functions RIOT-OS/riotdocker#95

Merged

benpicco approved these changes Jan 11, 2020

View reviewed changes

benpicco merged commit 3672502 into RIOT-OS:master Jan 15, 2020

fjmolinas added this to the Release 2020.01 milestone Jan 15, 2020

gschorcht deleted the cpu/esp/common_sdk_log_output branch January 16, 2020 13:15

gschorcht mentioned this pull request Jan 31, 2020

cpu/esp*: remove dependencies on GNRC for ESP network device drivers #12967

Merged

3 tasks

gschorcht mentioned this pull request Mar 5, 2020

[TRACKING] cpu/esp8266: Open problems of the ESP8266 port based on the ESP8266 RTOS SDK #12707

Open

9 tasks

Conversation

gschorcht commented Dec 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Testing procedure

Issues/PRs references

Uh oh!

gschorcht commented Dec 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gschorcht commented Jan 7, 2020

Uh oh!

benpicco commented Jan 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gschorcht commented Jan 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gschorcht commented Jan 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gschorcht commented Jan 14, 2020

Uh oh!

gschorcht commented Jan 15, 2020

Uh oh!

benpicco commented Jan 15, 2020

Uh oh!

fjmolinas commented Jan 15, 2020

Uh oh!

fjmolinas commented Jan 15, 2020

Uh oh!

gschorcht commented Jan 15, 2020

Uh oh!

fjmolinas commented Jan 15, 2020

Uh oh!

gschorcht commented Jan 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gschorcht commented Dec 20, 2019 •

edited

Loading

gschorcht commented Dec 20, 2019 •

edited

Loading

benpicco commented Jan 11, 2020 •

edited

Loading

gschorcht commented Jan 12, 2020 •

edited

Loading

gschorcht commented Jan 12, 2020 •

edited

Loading