Skip to content

cpu/esp8266: Fix crashes of the ESP866 when rebooting with or disconnecting from WiFi#22014

Merged
leandrolanzieri merged 2 commits intoRIOT-OS:masterfrom
gschorcht:cpu/esp_common/fix_freertos_task_delay
Jan 27, 2026
Merged

cpu/esp8266: Fix crashes of the ESP866 when rebooting with or disconnecting from WiFi#22014
leandrolanzieri merged 2 commits intoRIOT-OS:masterfrom
gschorcht:cpu/esp_common/fix_freertos_task_delay

Conversation

@gschorcht
Copy link
Copy Markdown
Contributor

Contribution description

This PR fixes the problem described in issue #21558.

With the ESP8266, vTaskDelay is only called by ieee80211_sta_new_state, with interrupts of all levels disabled, for example as a result of executing esp_wifi_disconnect. vTaskDelay uses ztimer_sleep and if ztimer_sleep is then called with interrupts disabled, it leads to memory corruption for some reason when interrupts are re-enabled afterwards. The only way to avoid this is to re-enable interrupts before calling ztimer_sleep in vTaskDelay.

Since debugging ESP8266 code is very limited and sometimes impossible due to there being only one hardware breakpoint and a lot of closed binary code involved when calling esp_wifi_disconnect, it was not possible to find the reason for the memory corruption. An exception occurs after executing esp_wifi_disconnect in ztimer_handler when
_callback_unlock_mutex is called with a mutex parameter that points to wrong address in IROM.

Therefore, enabling the interrupts in vTaskDelay before calling ztimer_sleep is currently the only way to avoid the exception, even though this is only a hack.

The PR includes a small improvement of the pm_reboot function. Even though esp_wifi_stop is implicitly called later in the restart process by esp_restart, it is cleaner to stop the WiFi interface here before saving the RTT counters.

Testing procedure

Compile and flash the GNRC networking example for any ESP8266 with WiFi enabled:

CFLAGS='-DWIFI_SSID=\"ssid\" -DWIFI_PASS=\"pass\"' USEMODULE='esp_wifi' \
BOARD=esp8266-esp-12x make -j8 -C examples/networking/gnrc/networking flash term

Without the PR, using reboot after connecting to the WiFi AP will lead to an exception. With the PR, it should work.

Issues/PRs references

Fixes #21558

With the ESP8266, this function is only called by
`ieee80211_sta_new_state`, with interrupts of all levels disabled,
for example as a result of executing `esp_wifi_disconnect`.
If `ztimer_sleep` is then called with interrupts disabled, for
some reason this leads to memory corruption when interrupts are
re-enabled afterwards. The only way to avoid this is to re-enable
interrupts before calling `ztimer_sleep`.

Since debugging ESP8266 code is very limited and sometimes impossible
due to there being only one hardware breakpoint and a lot of closed
binary code involved when calling `esp_wifi_disconnect`, it was not
possible to find the reason for the memory corruption. An exception
occurs after executing `esp_wifi_disconnect` in `ztimer_handler` when
`_callback_unlock_mutex` is called with a mutex parameter that points
to irrelevant read-only memory in IROM.

Therefore, enabling the interrupts here is currently the only way to
avoid the exception, even though this is only a hack.
Even though `esp_wifi_stop` is implicitly called later in the restart process by `esp_restart`, it is cleaner to stop the WiFi interface here before saving the RTT counters.
@gschorcht gschorcht requested a review from kaspar030 as a code owner January 25, 2026 09:56
@github-actions github-actions bot added Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: cpu Area: CPU/MCU ports labels Jan 25, 2026
@gschorcht gschorcht requested a review from benpicco January 25, 2026 09:56
@gschorcht gschorcht added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jan 25, 2026
@gschorcht gschorcht force-pushed the cpu/esp_common/fix_freertos_task_delay branch from a998f55 to e2d0e28 Compare January 25, 2026 09:59
@riot-ci
Copy link
Copy Markdown

riot-ci commented Jan 25, 2026

Murdock results

✔️ PASSED

e2d0e28 cpu/esp8266/periph: call esp_wifi_stop in pm_reboot

Success Failures Total Runtime
11003 0 11004 17m:40s

Artifacts

@leandrolanzieri
Copy link
Copy Markdown
Contributor

@gschorcht should we also backport this?

@gschorcht
Copy link
Copy Markdown
Contributor Author

I don't know how often this actually happens. To be honest, I wasn't aware of this problem before. @benpicco What do you think?

@leandrolanzieri
Copy link
Copy Markdown
Contributor

I just tested the fix and can confirm it works for me.

Without the PR
Type '/exit' to exit.
2026-01-27 14:38:59,005 # 
2026-01-27 14:38:59,005 # 
2026-01-27 14:38:59,006 # NETOPT_TX_END_IRQ not implemented by driver
2026-01-27 14:38:59,007 # main(): This is RIOT! (Version: 2026.04-devel-33-g0a609)
2026-01-27 14:38:59,007 # RIOT network stack example application
2026-01-27 14:38:59,007 # All up, running the shell now
> ifconfig
2026-01-27 14:39:00,858 # ifconfig
2026-01-27 14:39:00,864 # Iface  10  HWaddr: 84:F3:EB:29:17:71  Channel: 0  Link: down 
2026-01-27 14:39:00,865 #           L2-PDU:1500  MTU:1500  HL:64  RTR  
2026-01-27 14:39:00,870 #           RTR_ADV  
2026-01-27 14:39:00,870 #           Source address length: 6
2026-01-27 14:39:00,875 #           Link type: wireless
2026-01-27 14:39:00,876 #           inet6 group: ff02::1a
2026-01-27 14:39:00,876 #           
2026-01-27 14:39:00,881 #           Statistics for Layer 2
2026-01-27 14:39:00,881 #             RX packets 0  bytes 0
2026-01-27 14:39:00,887 #             TX packets 0 (Multicast: 0)  bytes 0
2026-01-27 14:39:00,892 #             TX succeeded 0 errors 0
2026-01-27 14:39:00,892 #           Statistics for IPv6
2026-01-27 14:39:00,897 #             RX packets 0  bytes 0
2026-01-27 14:39:00,898 #             TX packets 0 (Multicast: 0)  bytes 0
2026-01-27 14:39:00,903 #             TX succeeded 0 errors 0
2026-01-27 14:39:00,903 # 
> 2026-01-27 14:39:01,172 # WiFi connected to ssid RIOT-HIL, channel 6
> reboot
2026-01-27 14:39:58,782 # reboot
2026-01-27 14:39:58,790 # WiFi disconnected from ssid RIOT-HIL, reason 8 (ASSOC_LEAVE)
2026-01-27 14:39:58,892 # EXCEPTION!! exccause=9 (LoadStoreAlignmentCause) @402106e1 excvaddr=4024c9ff
2026-01-27 14:39:58,893 # processes:
2026-01-27 14:39:58,898 #       pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2026-01-27 14:39:58,909 #         - | isr_stack            | -        - |   - |   2048 ( 1076) (  972) | 0x3ffe8000 | 0x3ffe8800
2026-01-27 14:39:58,920 #         1 | ppT                  | bl mutex _ |   2 |   3640 ( 1320) ( 2320) | 0x3fff67e0 | 0x3fff7480 
2026-01-27 14:39:58,926 #         2 | pmT                  | bl mutex _ |   4 |   1080 (  380) (  700) | 0x3fff7660 | 0x3fff7930 
2026-01-27 14:39:58,937 #         3 | rtT                  | bl mutex _ |   3 |   2104 ( 1100) ( 1004) | 0x3fff7aa0 | 0x3fff8170 
2026-01-27 14:39:58,943 #         4 | esp_events           | bl mutex _ |   5 |   2104 (  892) ( 1212) | 0x3fff8910 | 0x3fff8fb0 
2026-01-27 14:39:58,953 #         5 | idle                 | running  Q |  31 |   1024 (  236) (  788) | 0x3ffee218 | 0x3ffee540 
2026-01-27 14:39:58,959 #         6 | main                 | bl mutex _ |  15 |   3072 ( 1644) ( 1428) | 0x3ffee618 | 0x3ffeefb0 
2026-01-27 14:39:58,970 #         7 | pktdump              | bl rx    _ |  14 |   2048 (  316) ( 1732) | 0x3fff34d8 | 0x3fff3bb0 
2026-01-27 14:39:58,976 #         8 | ipv6                 | bl rx    _ |  12 |   1536 (  552) (  984) | 0x3fff1298 | 0x3fff1750 
2026-01-27 14:39:58,987 #         9 | udp                  | bl rx    _ |  13 |   1472 (  360) ( 1112) | 0x3fff4374 | 0x3fff47e0 
2026-01-27 14:39:58,992 #        10 | netif-esp-wifi       | bl anyfl _ |  10 |   1536 (  852) (  684) | 0x3ffef880 | 0x3ffefd60 
2026-01-27 14:39:59,004 #        11 | RPL                  | bl rx    _ |  13 |   1024 (  340) (  684) | 0x3fff3f30 | 0x3fff41f0 
2026-01-27 14:39:59,009 #           | SUM                  |            |     |  22688 ( 9068) (13620)
2026-01-27 14:39:59,009 # 
2026-01-27 14:39:59,015 # /data/riotbuild/riotbase/cpu/esp_common/syscalls.c:203 => *** RIOT kernel panic:
2026-01-27 14:39:59,020 # FAILED ASSERTION.
2026-01-27 14:39:59,020 # 
2026-01-27 14:39:59,026 #       pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2026-01-27 14:39:59,037 #         - | isr_stack            | -        - |   - |   2048 ( 1252) (  796) | 0x3ffe8000 | 0x3ffe8800
2026-01-27 14:39:59,042 #         1 | ppT                  | bl mutex _ |   2 |   3640 ( 1320) ( 2320) | 0x3fff67e0 | 0x3fff7480 
2026-01-27 14:39:59,053 #         2 | pmT                  | bl mutex _ |   4 |   1080 (  380) (  700) | 0x3fff7660 | 0x3fff7930 
2026-01-27 14:39:59,059 #         3 | rtT                  | bl mutex _ |   3 |   2104 ( 1100) ( 1004) | 0x3fff7aa0 | 0x3fff8170 
2026-01-27 14:39:59,070 #         4 | esp_events           | bl mutex _ |   5 |   2104 (  892) ( 1212) | 0x3fff8910 | 0x3fff8fb0 
2026-01-27 14:39:59,081 #         5 | idle                 | running  Q |  31 |   1024 (  236) (  788) | 0x3ffee218 | 0x3ffee540 
2026-01-27 14:39:59,087 #         6 | main                 | bl mutex _ |  15 |   3072 ( 1644) ( 1428) | 0x3ffee618 | 0x3ffeefb0 
2026-01-27 14:39:59,097 #         7 | pktdump              | bl rx    _ |  14 |   2048 (  316) ( 1732) | 0x3fff34d8 | 0x3fff3bb0 
2026-01-27 14:39:59,103 #         8 | ipv6                 | bl rx    _ |  12 |   1536 (  552) (  984) | 0x3fff1298 | 0x3fff1750 
2026-01-27 14:39:59,114 #         9 | udp                  | bl rx    _ |  13 |   1472 (  360) ( 1112) | 0x3fff4374 | 0x3fff47e0 
2026-01-27 14:39:59,120 #        10 | netif-esp-wifi       | bl anyfl _ |  10 |   1536 (  852) (  684) | 0x3ffef880 | 0x3ffefd60 
2026-01-27 14:39:59,131 #        11 | RPL                  | bl rx    _ |  13 |   1024 (  340) (  684) | 0x3fff3f30 | 0x3fff41f0 
2026-01-27 14:39:59,137 #           | SUM                  |            |     |  22688 ( 9244) (13444)
2026-01-27 14:39:59,137 # 
2026-01-27 14:39:59,137 # *** halted.
2026-01-27 14:39:59,137 # 

With this PR
Type '/exit' to exit.
2026-01-27 14:43:59,428 # 
2026-01-27 14:43:59,428 # NETOPT_TX_END_IRQ not implemented by driver
2026-01-27 14:43:59,430 # main(): This is RIOT! (Version: 2026.04-devel-31-ge2d0e-cpu/esp_common/fix_freertos_task_delay)
2026-01-27 14:43:59,430 # RIOT network stack example application
2026-01-27 14:43:59,431 # All up, running the shell now
> 2026-01-27 14:44:01,569 # WiFi connected to ssid RIOT-HIL, channel 6
ifconfig
2026-01-27 14:44:19,854 # ifconfig
2026-01-27 14:44:19,859 # Iface  10  HWaddr: 84:F3:EB:29:17:71  Channel: 6  Link: up 
2026-01-27 14:44:19,860 #           L2-PDU:1500  MTU:1500  HL:64  RTR  
2026-01-27 14:44:19,865 #           RTR_ADV  
2026-01-27 14:44:19,865 #           Source address length: 6
2026-01-27 14:44:19,870 #           Link type: wireless
2026-01-27 14:44:19,876 #           inet6 addr: fe80::86f3:ebff:fe29:1771  scope: link  VAL
2026-01-27 14:44:19,882 #           inet6 addr: fd68:cdb2:4ca4:0:86f3:ebff:fe29:1771  scope: global  VAL
2026-01-27 14:44:19,882 #           inet6 group: ff02::1a
2026-01-27 14:44:19,887 #           inet6 group: ff02::2
2026-01-27 14:44:19,887 #           inet6 group: ff02::1
2026-01-27 14:44:19,893 #           inet6 group: ff02::1:ff29:1771
2026-01-27 14:44:19,893 #           
2026-01-27 14:44:19,898 #           Statistics for Layer 2
2026-01-27 14:44:19,899 #             RX packets 34  bytes 4350
2026-01-27 14:44:19,904 #             TX packets 5 (Multicast: 3)  bytes 430
2026-01-27 14:44:19,909 #             TX succeeded 5 errors 0
2026-01-27 14:44:19,910 #           Statistics for IPv6
2026-01-27 14:44:19,915 #             RX packets 32  bytes 3156
2026-01-27 14:44:19,920 #             TX packets 5 (Multicast: 3)  bytes 360
2026-01-27 14:44:19,920 #             TX succeeded 5 errors 0
2026-01-27 14:44:19,920 # 
> reboot
2026-01-27 14:44:41,780 # reboot
2026-01-27 14:44:41,888 # WiFi disconnected from ssid RIOT-HIL, reason 8 (ASSOC_LEAVE)
2026-01-27 14:44:42,214 # rll��|�l�


                                   �
                                    l�b|����r�bl
                                                b��n�lnn��
                                                          bp��lrlrlp��

                                                                      �
                                                                       l





                                                                        b
                                                                         n�
                                                                           ����b��n�l��l`�nn
                                                                                            l`nr���n

                                                                                                    bl�`p��

                                                                                                           r����



                                                                                                                b
                                                                                                                 n�l

                                                                                                                    �
                                                                                                                     b��n�

                                                                                                                          l`�nn
                                                                                                                               l`nr���n

                                                                                                                                       rrl`r��n

                                                                                                                                               rr��
2026-01-27 14:44:42,214 # 
2026-01-27 14:44:42,256 # NETOPT_TX_END_IRQ not implemented by driver
2026-01-27 14:44:42,266 # main(): This is RIOT! (Version: 2026.04-devel-31-ge2d0e-cpu/esp_common/fix_freertos_task_delay)
2026-01-27 14:44:42,267 # RIOT network stack example application
2026-01-27 14:44:42,270 # All up, running the shell now
> 2026-01-27 14:44:45,173 # WiFi connected to ssid RIOT-HIL, channel 6

@leandrolanzieri leandrolanzieri added the Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch label Jan 27, 2026
@leandrolanzieri leandrolanzieri added this to the Release 2026.01 milestone Jan 27, 2026
@leandrolanzieri leandrolanzieri added this pull request to the merge queue Jan 27, 2026
@leandrolanzieri
Copy link
Copy Markdown
Contributor

Let's go with it then

Merged via the queue into RIOT-OS:master with commit 8488b9a Jan 27, 2026
27 checks passed
@leandrolanzieri
Copy link
Copy Markdown
Contributor

@gschorcht let's also backport this

@gschorcht
Copy link
Copy Markdown
Contributor Author

Thanks for reviewing and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: cpu Area: CPU/MCU ports CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Platform: ESP Platform: This PR/issue effects ESP-based platforms Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

esp8266 crashes on WiFi disconnect

4 participants