I'm getting an error compiling tests/kconfig for native:

Building application "tests_kconfig" for "native" with MCU "native".

Traceback (most recent call last):
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/merge_config.py", line 82, in <module>
    kconf = Kconfig(sys.argv[1])
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 1056, in __init__
    self._parse_block(None, self.top_node, self.top_node)
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 2931, in _parse_block
    prev = self._parse_block(None, parent, prev)
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 2856, in _parse_block
    while self._next_line():
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 2183, in _next_line
    self._tokens = self._tokenize(line)
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 2467, in _tokenize
    self._parse_error("unknown tokens in line")
  File "/home/kaspar/src/riot.tmp/dist/tools/kconfiglib/kconfiglib.py", line 3820, in _parse_error
    raise KconfigError("{}couldn't parse '{}': {}".format(
kconfiglib.KconfigError: bin/native/generated/Kconfig.dep:1: couldn't parse 'config MODULE_AUTO_INIT\NBOARD\NCORE\NCORE_MSG\NCPU\NNATIVE_DRIVERS\NPERIPH\NPERIPH_COMMON\NPERIPH_GPIO\NPERIPH_PM\NPERIPH_UART\NSYS': unknown tokens in line
make: *** [/home/kaspar/src/riot.tmp/makefiles/kconfig.mk:72: /home/kaspar/src/riot.tmp/tests/kconfig/bin/native/generated/merged.config] Error 1

I'm getting an error compiling tests/kconfig for native:

The good old 'it works on my machine' :-) . This should be more portable now.

This compiles now just fine.

With an empty bin/ folder, I can do "make menuconfig", disable msg2, change msg one, do "make" and see the results.
If I now do "make menuconfig" again, re-enabling msg2, re-doing "make", the built binary still shows message one. Is that intended?

This compiles now just fine.

Nice! Thanks for confirming.

With an empty bin/ folder, I can do "make menuconfig", disable msg2, change msg one, do "make" and see the results.
If I now do "make menuconfig" again, re-enabling msg2, re-doing "make", the built binary still shows message one. Is that intended?

After re-enabling msg2 it shows up again (together with msg1), right? If that is the case then it is correct. msg1 and msg2 are independent.

The idea was to show with msg1 how configuration options would be handled in modules. When you enable "Configure message 1" (KCONFIG_APP_MSG_1), as the help string mentions, what you are doing is enabling the configuration of that message via Kconfig. So it will allow the user to change the text of msg1 via Kconfig symbol APP_MSG_1_TEXT. When that is not enabled then the default text from the header file will be used.

In short, msg1 is meant to be always on, you can only modify the text itself.
On the other hand, msg2 actually allows to enable/disable the message.

After re-enabling msg2 it shows up again (together with msg1), right?

No, it doesn't. I think just doing "make" doesn't rebuild properly.

Like this:

1. cd tests/kconfig
2. rm -Rf bin
3. make
4. ... run .elf, see both message 1 and MSG_2
5. make menuconfig, disable MSG2, save
6. make
7. ... run .elf, MSG_2 is still shown.

I usually always build with "make clean all", but IIRC this will clean bin/ and thus the changed configuration.

5\. `make menuconfig`, disable MSG2, save

I can confirm that "bin/native/generated/merged.config" contains # CONFIG_APP_MSG_2 is not set.

5\. `make menuconfig`, disable MSG2, save

I can confirm that "bin/native/generated/merged.config" contains # CONFIG_APP_MSG_2 is not set.

Is CONFIG_APP_MSG_2 being defined in bin/native/generated/autoconf.h?

Is CONFIG_APP_MSG_2 being defined in bin/native/generated/autoconf.h?

no, not present there.

Is CONFIG_APP_MSG_2 being defined in bin/native/generated/autoconf.h?

no, not present there.

Ok, so the header file is being generated correctly. I don't see why the binary would not be updated as autoconf.h is dependency, and it's changing. Can you confirm that the last-change date of the binary stays the same after changing autoconf.h?

After re-enabling msg2 it shows up again (together with msg1), right?

No, it doesn't. I think just doing "make" doesn't rebuild properly.

Like this:

1. cd tests/kconfig
2. rm -Rf bin
3. make
4. ... run .elf, see both message 1 and MSG_2
5. make menuconfig, disable MSG2, save
6. make
7. ... run .elf, MSG_2 is still shown.

I usually always build with "make clean all", but IIRC this will clean bin/ and thus the changed configuration.

I can't reproduce the issue. I followed the steps and after disabling msg2 the .elf is updated and the message is gone.

I can't reproduce the issue. I followed the steps and after disabling msg2 the .elf is updated and the message is gone.

If I disable ccache ($ unset CCACHE), this works as expected (msg_2 gone in step 7.).

If I disable ccache ($ unset CCACHE), this works as expected (msg_2 gone in step 7.).

I don't understand why.
touch bin/native/generated/autoconf.h && make QUIET=0 correctly rebuilds everything, if ccache is not used. With ccache enabled, ccache/gcc is not even called, so ccache is not returning a wrong cache result.

I think the issue is that ccache transform absolute path's to relative paths within the dependency file, which make doesn't seem to match properly. So probably, with ccache, the automatic dependencies were broken before. With ccache, always rebuilding with "clean" isn't nearly as slow as depending on make's dependency resolution, so noone realized.

Problem is that the kconfig user configuration sits in "bin/" (and gets cleared with "make clean"), so as of now, it's either kconfig or ccache.

I think the issue is that ccache transform absolute path's to relative paths within the dependency file, which make doesn't seem to match properly.

Hm, but touching riotbuild.h properly causes a rebuild independent of ccache enabled or not.

I think the issue is that ccache transform absolute path's to relative paths within the dependency file, which make doesn't seem to match properly.

Hm, but touching riotbuild.h properly causes a rebuild independent of ccache enabled or not.

I now enabled ccache and can confirm that the binary is not being updated. I managed to get the application re-compiled by manually adding the autoconf.h file to ccache 'extra_files_to_hash' configuration. So I think it's as you say, there is some dependency problem with ccache.

@kaspar030 I added autoconf.h as a dependency for the object files, so now when the configurations are changed the binaries should be re-compiled.

I rebased to current master, now that #12695 is merged.

BTW you can limit what murdock builds by adding a "REMOVEME" commit that sets e.g., "APPS=examples/hello-world" and/or "BOARDS=samr21-xpro" in .murdock, so only a subset is built. That helps improving iteration time for figuring out a specific issue.

I can easily trigger the issue locally with BOARD=nrf52dk make -Ctests/pkg_microcoap clean all -j

Nice! Thanks @kaspar030, that worked!

Murdock is happy now! @kaspar030 anything else that is missing?

Murdock is happy now! @kaspar030 anything else that is missing?

Nope, LGTM.

@leandrolanzieri A short question. At the moment, it is possible to define menu items in Kconfig depending on enabled modules. Would it also be possible to enable a module for compilation by enabling an option in Kconfig? For example:

    ESP32 --->
        [  ] ESP-WiFi settings
        [  ] ESP-NOW setting

Enabling the ESP-WiFi settings would enter into the configuration menu for ESP-WiFi and would add USEMODULE += esp_wifi to the make configuration.

The background for my question is that a lot of configurations are done in RIOT using pseudomodules. Now, where we have a menu driven way for configuration, it would be nice to be able to enable or disable also modules.

@leandrolanzieri One further question. I'm working on exposing ESP configurations with Kconfig. In all your PRs, you use menuconfig KCONFIG_MODULE_XYZ instead of menu "XYZ" for sub-categories. This has the disadvantage that the user must first select the subcategory before he can enter the configuration.

Is this the desired behavior? It feels a little strange from the user's perspective. It suggests that this enables the module.

Are there any design rules or coding conventions, or could I also use menu "XYZ"?

@gschorcht

Would it also be possible to enable a module for compilation by enabling an option in Kconfig?

Not at the moment, but that's part of the plan for the second phase of migration (to resolve dependencies via Kconfig). The problem is that modules should be resolved by the time the Kconfig configurator is called. If we enable modules via Kconfig that may result in new modules being enabled, so the input Kconfig had may no longer be valid.

I'm working on exposing ESP configurations with Kconfig

Thanks for working on this!

Is this the desired behavior? It feels a little strange from the user's perspective. It suggests that this enables the module.

This is the intended behavior. During this phase of migration we are making the configuration of modules via Kconfig optional (that's why prompts say 'Configure ', and help strings also mention this). In the future though, the idea is that it actually enables the module as you suggest.

I wrote some documentation in #13040, there you can find more background and some conventions.

@leandrolanzieri Thanks for the explanation.

This is the intended behavior. During this phase of migration we are making the configuration of modules via Kconfig optional (that's why prompts say 'Configure ', and help strings also mention this). In the future though, the idea is that it actually enables the module as you suggest.

OK, then I will do it in same way.

I wrote some documentation in #13040, there you can find more background and some conventions.

I will have look on it later this week.