Skip to content

MuleDebug: PIE-translate addresses for the addr2line fallback too - #678

Merged
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/muledebug-addr2line-pie
May 22, 2026
Merged

MuleDebug: PIE-translate addresses for the addr2line fallback too#678
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/muledebug-addr2line-pie

Conversation

@got3nks

@got3nks got3nks commented May 22, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #677, reported by @danim7.

#677 fixed the libbfd path's PIE handling but left the no-libbfd fallback broken in exactly the same way: it shells out to addr2line -e /proc/<pid>/exe <addrs> with raw runtime PCs, which addr2line can't resolve for PIE binaries. After uninstalling binutils-dev, @danim7 confirmed amule frames came back as ?? -- same symptom as the pre-#677 bfd path.

How

Hoist the s_pie_base plumbing (#include <link.h>, find_pie_base_cb, dl_iterate_phdr call) out of #ifdef HAVE_BFD into the shared Linux scope. Wrap the lookup in init_pie_base() (idempotent -- no-op on second call) so the bfd path's init_backtrace_info() and the no-bfd path of get_backtrace() both seed it at the right moment without duplicating the dl_iterate_phdr call.

In the no-bfd branch, build a translatedAddresses string by subtracting s_pie_base from each runtime PC before formatting into the addr2line command line. PIE binaries get the link-time addresses addr2line expects. Non-PIE binaries have s_pie_base = 0 (dlpi_addr is 0 for ET_EXEC) so the subtraction is a no-op and the addr2line invocation is byte-identical to before -- no regression possible on the non-PIE path.

Verification

  • Linux Debug + libbfd (regression): clean build, behaviour unchanged.
  • Linux Debug + HAVE_BFD forced FALSE (exercises addr2line path with the new translation): clean build.
  • macOS (path is #ifdef __APPLE__, unaffected): clean build.

Symmetric to #677. Both paths now use the same offset under the same conditions; either both work or both break on the same input.

@danim7

danim7 commented May 22, 2026

Copy link
Copy Markdown
Contributor

Mmmmh, not working on my Debian 13.4, no backtrace is printed, the program just exists:

user@debian:~/amule/got3nks/amule$ ./build/src/amule                          
 2026-05-22 17:36:42: amuleAppCommon.cpp(386): Initialising aMule GIT compiled with wxGTK3 v3.2.8 and Boost 1.83 (Debugging) (Snapshot: rev. 2.3.3-438-ga59899231)
 2026-05-22 17:36:42: amuleAppCommon.cpp(433): Checking if there is an instance already running...
 2026-05-22 17:36:42: amuleAppCommon.cpp(464): No other instances are running.
 2026-05-22 17:36:42: ListenSocket.cpp(63): ListenSocket: Ok.
 2026-05-22 17:36:42: DownloadQueue.cpp(109): Loading temp files from /home/user/.aMule/Temp.
 2026-05-22 17:36:42: DownloadQueue.cpp(159): All PartFiles Loaded.
 2026-05-22 17:36:42: LibSocketAsio.cpp(1486): Asio thread 1 started
 2026-05-22 17:36:42: LibSocketAsio.cpp(1486): Asio thread 3 started
 2026-05-22 17:36:42: LibSocketAsio.cpp(1486): Asio thread 2 started
 2026-05-22 17:36:42: LibSocketAsio.cpp(1486): Asio thread 4 started
Trace/breakpoint trap
user@debian:~/amule/got3nks/amule$ 

@got3nks
got3nks marked this pull request as draft May 22, 2026 17:40
@got3nks

got3nks commented May 22, 2026

Copy link
Copy Markdown
Contributor Author

@danim7 -- I reproduced your exact wxASSERT(false) in OnCoreTimer on an aarch64 Ubuntu with HAVE_BFD forced FALSE (so the addr2line path is the live one), and amuled produced a fully symbolicated backtrace with function names + line numbers, no SIGTRAP. So the no-bfd path itself isn't intrinsically broken with #678 -- something is different about Debian 13.4 / x86_64 / the GUI amule binary specifically.

Without your gdb output I can only guess. Could you run:

gdb --args ./build/src/amule
(gdb) handle SIGTRAP nostop noprint pass    # to skip any benign SIGTRAPs from threads
(gdb) run
(when "Trace/breakpoint trap" hits)
(gdb) bt
(gdb) info reg pc
(gdb) x/8i $pc-16
(gdb) quit

The bt will tell us which function trapped. x/8i $pc-16 decodes the instructions around the trap PC -- specifically I want to see whether it's a ud2 (compiler-emitted __builtin_trap, often from UBSan or __builtin_unreachable() reached at runtime) or an int3 (debug break / glibc fortify failure).

Two specific theories worth ruling in/out from your build:

  1. Did your Debian package something that adds -fsanitize=... or -D_FORTIFY_SOURCE=3 to the default CMAKE_CXX_FLAGS_DEBUG? Run grep -E "fsanitize|FORTIFY" /etc/dpkg/origins/* /etc/dpkg/buildflags.conf 2>/dev/null and cat build/CMakeFiles/CMakeCache.txt | grep -E "CXX_FLAGS|C_FLAGS". If either fortify or sanitizer hooks are active, SIGTRAP is the expected signal for several check failures (vs. SIGABRT for plain abort()).
  2. Does the same wxASSERT(false) repro on Debian 13.4 work on plain master (without MuleDebug: PIE-translate addresses for the addr2line fallback too #678 applied)? git checkout origin/master && rebuild && run. If master also SIGTRAPs, this is upstream of my changes.

Worth doing both before chasing #678 specifically, because nothing in #678's no-bfd code path looks like it could produce a raw SIGTRAP -- the failure mode I'd expect from a bug in my code would be SIGSEGV or an unhandled exception, not SIGTRAP.

@danim7

danim7 commented May 22, 2026

Copy link
Copy Markdown
Contributor

Thanks @got3nks for your details instructions.

I will test and report back later today.

same PIE bug.  Reported by danim7 on the amule-project#677 thread after
they uninstalled libbfd to test the fallback: amule frames
came back as "??", and running addr2line manually with the
same arguments produced the same empty output -- the addr2line
tool expects link-time addresses, but the fallback hands it
runtime PCs.

Hoist the s_pie_base + find_pie_base_cb + dl_iterate_phdr
plumbing out of the #ifdef HAVE_BFD block into the shared
Linux scope so both paths can use it.  Wrap the actual lookup
in init_pie_base() (idempotent, no-op on subsequent calls) so
the bfd path's init_backtrace_info() and the no-bfd path of
get_backtrace() both seed it at the right moment without
duplicating the dl_iterate_phdr call.

In the no-bfd branch, translate each runtime PC by subtracting
s_pie_base before formatting into the addr2line command line.
For PIE binaries this yields the link-time address addr2line
expects.  For non-PIE binaries s_pie_base is 0 (dlpi_addr is
0 for ET_EXEC) so the subtraction is a no-op and the addr2line
invocation is byte-identical to before -- no regression
possible on the non-PIE path.

Symmetric to amule-project#677.  The two paths now use the same offset
under the same conditions and would either both work or both
break on the same kind of input.
@got3nks
got3nks force-pushed the fix/muledebug-addr2line-pie branch from a598992 to f41cedf Compare May 22, 2026 20:36
@got3nks
got3nks marked this pull request as ready for review May 22, 2026 20:36
@mrjimenez
mrjimenez merged commit 366dabb into amule-project:master May 22, 2026
12 checks passed
@danim7

danim7 commented May 22, 2026

Copy link
Copy Markdown
Contributor

Somehow, the first gdb step about handling SIGTRAP makes the program not to run:

user@debian:~/amule/got3nks/amule$ gdb --args ./build/src/amule 
GNU gdb (Debian 16.3-1) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/src/amule...
(gdb) handle SIGTRAP nostop noprint pass
SIGTRAP is used by the debugger.
Are you sure you want to change it? (y or n) y
Signal        Stop	Print	Pass to program	Description
SIGTRAP       No	No	Yes		Trace/breakpoint trap
(gdb) run
Starting program: /home/user/amule/got3nks/amule/build/src/amule 

Program terminated with signal SIGTRAP, Trace/breakpoint trap.
The program no longer exists.
(gdb) bt
No stack.
(gdb) info reg pc
The program has no registers now.
(gdb) x/8i $pc-16
No registers.
(gdb) quit

If I omit the first step, I can run it with the following result:

user@debian:~/amule/got3nks/amule$ gdb --args ./build/src/amule 
GNU gdb (Debian 16.3-1) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/src/amule...
(gdb) r
Starting program: /home/user/amule/got3nks/amule/build/src/amule 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff37266c0 (LWP 35841)]
[New Thread 0x7ffff2f256c0 (LWP 35842)]
[New Thread 0x7ffff27246c0 (LWP 35843)]
[Detaching after vfork from child process 35845]
 2026-05-22 21:09:26: amuleAppCommon.cpp(386): Initialising aMule GIT compiled with wxGTK3 v3.2.8 and Boost 1.83 (Debugging) (Snapshot: rev. 2.3.3-438-ga59899231)
 2026-05-22 21:09:26: amuleAppCommon.cpp(433): Checking if there is an instance already running...
 2026-05-22 21:09:26: amuleAppCommon.cpp(464): No other instances are running.
[New Thread 0x7ffff1de56c0 (LWP 35847)]
[New Thread 0x7ffff15e46c0 (LWP 35848)]
 2026-05-22 21:09:26: ListenSocket.cpp(63): ListenSocket: Ok.
[New Thread 0x7ffff0de36c0 (LWP 35849)]
[New Thread 0x7fffdbfff6c0 (LWP 35850)]
[Thread 0x7ffff0de36c0 (LWP 35849) exited]
[New Thread 0x7fffdb7fe6c0 (LWP 35851)]
[Detaching after vfork from child process 35852]
[New Thread 0x7fffda8476c0 (LWP 35854)]
[New Thread 0x7fffda0466c0 (LWP 35855)]
[New Thread 0x7fffd98456c0 (LWP 35856)]
[New Thread 0x7fffd90446c0 (LWP 35857)]
[New Thread 0x7fffd88436c0 (LWP 35858)]
[New Thread 0x7fffbbfff6c0 (LWP 35859)]
[New Thread 0x7fffbb7fe6c0 (LWP 35860)]
[Thread 0x7fffbb7fe6c0 (LWP 35860) exited]
 2026-05-22 21:09:27: DownloadQueue.cpp(109): Loading temp files from /home/user/.aMule/Temp.
 2026-05-22 21:09:27: DownloadQueue.cpp(159): All PartFiles Loaded.
[New Thread 0x7fffbb7fe6c0 (LWP 35861)]
[New Thread 0x7fffbaffd6c0 (LWP 35862)]
 2026-05-22 21:09:27: LibSocketAsio.cpp(1486): Asio thread 1 started
 2026-05-22 21:09:27: LibSocketAsio.cpp(1486): Asio thread 2 started
 2026-05-22 21:09:27: LibSocketAsio.cpp(1486): Asio thread 3 started
 2026-05-22 21:09:27: LibSocketAsio.cpp(1486): Asio thread 4 started
[New Thread 0x7ffff0de36c0 (LWP 35863)]
[Detaching after fork from child process 35864]
[Thread 0x7ffff0de36c0 (LWP 35863) exited]

Thread 1 "amule" received signal SIGTRAP, Trace/breakpoint trap.
0x00007ffff72b3709 in ?? () from /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
(gdb) bt
#0  0x00007ffff72b3709 in ??? () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#1  0x00007ffff72b04f3 in wxOnAssert(char const*, int, char const*, char const*, char const*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#2  0x00005555556e1e7c in CamuleApp::OnCoreTimer (this=0x555555cad110) at /home/user/amule/got3nks/amule/src/amule.cpp:1334
#3  0x00007ffff7437e6f in wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) ()
    at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#4  0x00007ffff7438043 in wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#5  0x00007ffff74386ad in wxEvtHandler::TryHereOnly(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#6  0x00007ffff743872e in wxEvtHandler::ProcessEventLocally(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#7  0x00007ffff7438831 in wxEvtHandler::ProcessEvent(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#8  0x00007ffff74399be in wxEvtHandler::SafelyProcessEvent(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#9  0x00007ffff7439ae4 in wxEvtHandler::ProcessPendingEvents() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#10 0x00007ffff72b067a in wxAppConsoleBase::ProcessPendingEvents() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#11 0x00007ffff6b6d755 in wxApp::DoIdle() () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#12 0x00007ffff6b6d837 in ??? () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#13 0x00007ffff7eab385 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#14 0x00007ffff7ead5b7 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#15 0x00007ffff7eae01f in g_main_loop_run () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#16 0x00007ffff540705d in gtk_main () at /lib/x86_64-linux-gnu/libgtk-3.so.0
#17 0x00007ffff6b8c405 in wxGUIEventLoop::DoRun() () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#18 0x00007ffff72ec901 in wxEventLoopBase::Run() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#19 0x00007ffff7403d6f in wxAppTraits::RunLoopUntilChildExit(wxExecuteData&, wxEventLoopBase&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#20 0x00007ffff6bc0407 in wxGUIAppTraits::WaitForChild(wxExecuteData&) () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#21 0x00007ffff740fa9a in wxExecute(char const* const*, int, wxProcess*, wxExecuteEnv const*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#22 0x00007ffff74109c0 in wxExecute(wxString const&, int, wxProcess*, wxExecuteEnv const*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#23 0x00007ffff743e42a in ??? () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#24 0x00005555559ca2fe in get_backtrace (n=2) at /home/user/amule/got3nks/amule/src/libs/common/MuleDebug.cpp:491
#25 0x00005555556e1929 in CamuleApp::OnAssertFailure
    (this=0x555555cad110, file=0x555556406b40 L"/home/user/amule/got3nks/amule/src/amule.cpp", line=1334, func=0x5555563db7d0 L"OnCoreTimer", cond=0x5555567e8060 L"false", msg=0x7fffffffda10 L"") at /home/user/amule/got3nks/amule/src/amule.cpp:1266
#26 0x00007ffff72b366f in ??? () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#27 0x00007ffff72b04f3 in wxOnAssert(char const*, int, char const*, char const*, char const*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#28 0x00005555556e1e7c in CamuleApp::OnCoreTimer (this=0x555555cad110) at /home/user/amule/got3nks/amule/src/amule.cpp:1334
#29 0x00007ffff7437e6f in wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) ()
    at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#30 0x00007ffff7438043 in wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#31 0x00007ffff74386ad in wxEvtHandler::TryHereOnly(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#32 0x00007ffff743872e in wxEvtHandler::ProcessEventLocally(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#33 0x00007ffff7438831 in wxEvtHandler::ProcessEvent(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#34 0x00007ffff74399be in wxEvtHandler::SafelyProcessEvent(wxEvent&) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
--Type <RET> for more, q to quit, c to continue without paging--
#35 0x00007ffff7439ae4 in wxEvtHandler::ProcessPendingEvents() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#36 0x00007ffff72b067a in wxAppConsoleBase::ProcessPendingEvents() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#37 0x00007ffff6b6d755 in wxApp::DoIdle() () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#38 0x00007ffff6b6d837 in ??? () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#39 0x00007ffff7eab385 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#40 0x00007ffff7ead5b7 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#41 0x00007ffff7eae01f in g_main_loop_run () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#42 0x00007ffff540705d in gtk_main () at /lib/x86_64-linux-gnu/libgtk-3.so.0
#43 0x00007ffff6b8c405 in wxGUIEventLoop::DoRun() () at /lib/x86_64-linux-gnu/libwx_gtk3u_core-3.2.so.0
#44 0x00007ffff72ec901 in wxEventLoopBase::Run() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#45 0x00007ffff72b1b2d in wxAppConsoleBase::MainLoop() () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#46 0x00007ffff733a03b in wxEntry(int&, wchar_t**) () at /lib/x86_64-linux-gnu/libwx_baseu-3.2.so.0
#47 0x00005555558180a1 in main (argc=1, argv=0x7fffffffe078) at /home/user/amule/got3nks/amule/src/amule-gui.cpp:98
(gdb) info reg pc
pc             0x7ffff72b3709      0x7ffff72b3709
(gdb) x/8i $pc-16
   0x7ffff72b36f9:	add    %eax,%eax
   0x7ffff72b36fb:	cld
   0x7ffff72b36fc:	ljmp   (bad)
   0x7ffff72b36fd:	jmp    0x7ffff72b362c
   0x7ffff72b3702:	nopw   0x0(%rax,%rax,1)
   0x7ffff72b3708:	int3
=> 0x7ffff72b3709:	mov    0x22ac7d(%rip),%eax        # 0x7ffff74de38c
   0x7ffff72b370f:	test   %eax,%eax
(gdb) quit
A debugging session is active.

	Inferior 1 [process 35838] will be killed.

Quit anyway? (y or n) y

Concerning your first question, nothing shows up:

user@debian:~/amule/got3nks/amule$ grep -E "fsanitize|FORTIFY" /etc/dpkg/origins/* /etc/dpkg/buildflags.conf 2>/dev/null
user@debian:~/amule/got3nks/amule$ cat build/CMakeFiles/CMakeCache.txt | grep -E "CXX_FLAGS|C_FLAGS"
cat: build/CMakeFiles/CMakeCache.txt: No such file or directory
user@debian:~/amule/got3nks/amule$ 

For the 2nd question, I retrieved version 543b716 and it shows the same behaviour:

./build/src/amule 
 2026-05-22 21:30:43: amuleAppCommon.cpp(386): Initialising aMule GIT compiled with wxGTK3 v3.2.8 and Boost 1.83 (Debugging) (Snapshot: rev. 2.3.3-440-g543b71600)
 2026-05-22 21:30:43: amuleAppCommon.cpp(433): Checking if there is an instance already running...
 2026-05-22 21:30:43: amuleAppCommon.cpp(464): No other instances are running.
 2026-05-22 21:30:44: ListenSocket.cpp(63): ListenSocket: Ok.
 2026-05-22 21:30:44: DownloadQueue.cpp(109): Loading temp files from /home/user/.aMule/Temp.
 2026-05-22 21:30:44: DownloadQueue.cpp(159): All PartFiles Loaded.
 2026-05-22 21:30:44: LibSocketAsio.cpp(1486): Asio thread 4 started
 2026-05-22 21:30:44: LibSocketAsio.cpp(1486): Asio thread 1 started
 2026-05-22 21:30:44: LibSocketAsio.cpp(1486): Asio thread 3 started
 2026-05-22 21:30:44: LibSocketAsio.cpp(1486): Asio thread 2 started
Trace/breakpoint trap

got3nks added a commit to got3nks/amule that referenced this pull request May 22, 2026
amule-project#678 follow-up reported by danim7.  The addr2line fallback in
get_backtrace() called wxExecute(cmd, out) to run addr2line.  In
GUI mode (amule), wxExecute on Linux waits for the child via
wxGUIAppTraits::WaitForChild which runs a nested wx event loop --
RunLoopUntilChildExit -> wxEventLoopBase::Run -> wxGUIEventLoop::
DoRun -> gtk_main -> g_main_loop_run -> wxApp::DoIdle ->
ProcessPendingEvents.

If the wxASSERT that brought us into get_backtrace() came from a
periodic / event-driven handler (OnCoreTimer, socket events,
etc.), the nested loop fires the same handler again, re-enters
wxASSERT, and the second-level wx assert handler can't run amule's
reentrantly so it falls through to wxTrap()'s int3 instruction,
delivering SIGTRAP and a "Trace/breakpoint trap" termination with
no backtrace printed at all.  Reproducer per danim7 in PR amule-project#678
discussion: wxASSERT(false) wired into CamuleApp::OnCoreTimer.

amuled doesn't reproduce because the console build's
wxExecute(cmd, out) uses a plain waitpid loop with no nested event
loop -- only the GUI build has the GUI-trait WaitForChild path.

Replace wxExecute with popen() / fgets() / pclose() -- a plain
fork+pipe+waitpid pipeline with no wx event-loop involvement.
The recursive-assert path simply doesn't exist anymore.  Output
parsing is identical (line-by-line into the same wxArrayString
that the old wxExecute populated).

Doesn't affect the libbfd path (which is in-process and doesn't
fork at all).  Recommended fix for users hitting wxASSERTs from
periodic handlers is still install binutils-dev so libbfd is the
live path; this PR ensures the addr2line fallback no longer
deadlocks-then-traps on the same input either.
@got3nks

got3nks commented May 22, 2026

Copy link
Copy Markdown
Contributor Author

@danim7 -- your gdb output nailed it. Recursive wxASSERT through wxExecute's nested wx event loop in the GUI build. Filed #682 as draft with the fix: replace wxExecute(cmd, out) with popen() / fgets() / pclose() in the addr2line fallback -- plain fork+waitpid, no wx event loop, no re-entrance path.

One thing we couldn't do locally though: I tried to reproduce your SIGTRAP on our aarch64 Ubuntu VM (wxGTK 3.2.9 + the exact same wxASSERT(false) in OnCoreTimer repro + HAVE_BFD=FALSE) and got a clean assertion popup + clean exit on both the pre-popen build (master + your repro) and the post-popen build -- no SIGTRAP either way. So your Debian 13.4 + wxGTK 3.2.8 setup is hitting the nested-loop recursion in a way wxGTK 3.2.9 isn't, possibly because wx changed something between those versions.

Could you cherry-pick #682 onto your tree and re-run your reproducer? If the trap is gone, we've confirmed it. If it's still there, the gdb backtrace from the new build will show whether the popen path even ran (and we'd need a different fix).

Either way the popen path is a strict structural improvement -- it removes the only place get_backtrace could nest a wx event loop, so the recursion vector is closed regardless of which wx version trips it.

mrjimenez pushed a commit that referenced this pull request May 22, 2026
#678 follow-up reported by danim7.  The addr2line fallback in
get_backtrace() called wxExecute(cmd, out) to run addr2line.  In
GUI mode (amule), wxExecute on Linux waits for the child via
wxGUIAppTraits::WaitForChild which runs a nested wx event loop --
RunLoopUntilChildExit -> wxEventLoopBase::Run -> wxGUIEventLoop::
DoRun -> gtk_main -> g_main_loop_run -> wxApp::DoIdle ->
ProcessPendingEvents.

If the wxASSERT that brought us into get_backtrace() came from a
periodic / event-driven handler (OnCoreTimer, socket events,
etc.), the nested loop fires the same handler again, re-enters
wxASSERT, and the second-level wx assert handler can't run amule's
reentrantly so it falls through to wxTrap()'s int3 instruction,
delivering SIGTRAP and a "Trace/breakpoint trap" termination with
no backtrace printed at all.  Reproducer per danim7 in PR #678
discussion: wxASSERT(false) wired into CamuleApp::OnCoreTimer.

amuled doesn't reproduce because the console build's
wxExecute(cmd, out) uses a plain waitpid loop with no nested event
loop -- only the GUI build has the GUI-trait WaitForChild path.

Replace wxExecute with popen() / fgets() / pclose() -- a plain
fork+pipe+waitpid pipeline with no wx event-loop involvement.
The recursive-assert path simply doesn't exist anymore.  Output
parsing is identical (line-by-line into the same wxArrayString
that the old wxExecute populated).

Doesn't affect the libbfd path (which is in-process and doesn't
fork at all).  Recommended fix for users hitting wxASSERTs from
periodic handlers is still install binutils-dev so libbfd is the
live path; this PR ensures the addr2line fallback no longer
deadlocks-then-traps on the same input either.
@got3nks
got3nks deleted the fix/muledebug-addr2line-pie branch May 22, 2026 23:44
mrjimenez pushed a commit that referenced this pull request May 23, 2026
…emoveFile

Issue #685 (JamesOlvertone): amuled crashed after renaming a file
in Incoming with shared-dir watching enabled.  Last log lines

  CFile: Error when opening file (/aMule/Incoming/foo.bar): No such ...
  Failed to open file (foo.bar), removing from list of shared files.

place the crash at CUploadDiskIOThread::StartCreateNextBlockPackage
calling theApp->sharedfiles->RemoveFile(srcfile) from a worker
thread.  Backtrace was addresses-only (build predated #678 +
binutils-dev not installed), but two structurally-real races on
m_keywords exist:

CSharedFileList::list_mut is the outer lock for both m_Files_map
and m_keywords.  m_keywords is a CPublishKeywordList holding a
std::list<CPublishKeyword*> + std::map<wxString, ...> and has no
internal locking; AddFile / RemoveFile / FindSharedFiles map-clear
all take list_mut around their m_keywords calls.  But:

1. CSharedFileList::Reload called m_keywords->RemoveAllKeywordReferences
   and m_keywords->PurgeUnreferencedKeywords without list_mut.  When
   the user renames a file under a watched dir, CSharedDirWatcher::
   OnFileSystemEvent fires ScheduleReload on the main thread.  If
   CUploadDiskIOThread is concurrently uploading that file, it sees
   CFile::Open fail and calls RemoveFile from the worker thread (the
   only RemoveFile call site that is not on the main thread).
   Worker holds list_mut while mutating m_keywords via RemoveKeywords;
   main thread iterates m_lstKeywords / m_keywordIndex / individual
   CPublishKeyword::references in parallel without the lock.  Result:
   iterator invalidation, ref-count race, or use-after-free of
   CPublishKeyword*.

2. CSharedFileList::Publish's keyword-publish section reads + mutates
   pPubKw (cursor advance via GetNextKeyword, GetReferences iteration,
   RotateReferences, SetNextPublishTime, IncPublishedCount) without
   list_mut.  Same worker race shape -- worker's RemoveKeyword
   decrements ref counts and removes pFile from pPubKw->references
   concurrently with main thread's read/iterate.

Wrap both call sites with list_mut.  Cannot lock the whole Reload
because FindSharedFiles walks the filesystem and would block the
worker pool for seconds; the lock is scoped tightly around the two
keyword ops there.  Publish's keyword section is wrapped wholesale --
Kad calls (PrepareLookup, StartSearch, Search::Go, GetClosestTo,
SendFindValue) do not re-enter list_mut, so holding the lock across
them is safe.  The source/notes publish sections (which call
GetCount + GetFileByIndex, both of which internally lock) are
intentionally left outside the new lock scope to avoid recursive
acquisition.
mrjimenez pushed a commit that referenced this pull request May 23, 2026
CamuleApp (amule.cpp) and CamuleDaemonApp both install a
wxApp::OnFatalException override that prints a libbfd/addr2line-
resolved in-process backtrace on SIGSEGV / SIGBUS / SIGABRT via
get_backtrace(). CamuleRemoteGuiApp doesn't, so when amulegui
crashes the wx default handler runs and exits without producing
any amule frames -- making diagnosis of GTK-callback-into-stale-
widget bugs (such as #692, search-tab close while results are
streaming) a guessing game even with a saved core, because the
core itself only sees gtk_main / libgtk frames and the systemd
coredump trace can't see into amule's process state at crash
time.

Mirror the existing CamuleApp::OnFatalException body in
CamuleRemoteGuiApp so amulegui's next crash dumps the same
backtrace amule(d) already do -- including the libbfd PIE fixes
from #677 / #678 that recently went in.

Body is intentionally a copy of CamuleApp::OnFatalException (only
the program name in the message differs) rather than refactored
into CamuleAppCommon: the latter is not a wxApp derivative so the
virtual override has to live on each wxApp-derived class anyway,
and a shared helper for the 18-line message string is not worth
the indirection.

#if wxUSE_ON_FATAL_EXCEPTION guard matches amule.cpp.
ngosang added a commit to ngosang/amule that referenced this pull request Jul 28, 2026
Translated using Weblate (Tamil)
Currently translated at 0.8% (2 of 246 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/ta/

Translated using Weblate

Translated using Weblate (Tamil)
Currently translated at 1.2% (24 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ta/

Translated using Weblate

Translated using Weblate (Latvian)
Currently translated at 56.9% (140 of 246 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/lv/

Translated using Weblate

Translated using Weblate (Latvian)
Currently translated at 29.8% (567 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/lv/

Translated using Weblate

Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 87.2% (218 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/zh_Hant/

Translated using Weblate

Translated using Weblate (Turkish)
Currently translated at 98.4% (246 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/tr/

Translated using Weblate

Translated using Weblate (Russian)
Currently translated at 87.2% (218 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/ru/

Translated using Weblate

Translated using Weblate (Romanian)
Currently translated at 87.2% (218 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/ro/

Translated using Weblate

Translated using Weblate (Portuguese (Brazil))
Currently translated at 97.2% (243 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/pt_BR/

Translated using Weblate

Translated using Weblate (Italian)
Currently translated at 100.0% (250 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/it/

Translated using Weblate

Translated using Weblate (Hungarian)
Currently translated at 87.2% (218 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/hu/

Translated using Weblate

Translated using Weblate (French)
Currently translated at 98.4% (246 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/fr/

Translated using Weblate

Translated using Weblate (Spanish)
Currently translated at 98.4% (246 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/es/

Translated using Weblate

Translated using Weblate (German)
Currently translated at 96.8% (242 of 250 strings)
Translation: aMule/Application Man Pages
Translate-URL: https://hosted.weblate.org/projects/amule/application-man-pages/de/

Translated using Weblate

Translated using Weblate (Chinese (Traditional Han script))
Currently translated at 81.4% (1549 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/zh_Hant/

Translated using Weblate

Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 97.4% (1852 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/zh_Hans/

Translated using Weblate

Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 97.4% (1852 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/zh_Hans/

Translated using Weblate

Translated using Weblate (Ukrainian)
Currently translated at 75.0% (1426 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/uk/

Translated using Weblate

Translated using Weblate (Turkish)
Currently translated at 97.1% (1847 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/tr/

Translated using Weblate

Translated using Weblate (Swedish)
Currently translated at 82.1% (1562 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/sv/

Translated using Weblate

Translated using Weblate (Albanian)
Currently translated at 61.1% (1163 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/sq/

Translated using Weblate

Translated using Weblate (Slovenian)
Currently translated at 82.3% (1566 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/sl/

Translated using Weblate

Translated using Weblate (Russian)
Currently translated at 81.2% (1545 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ru/

Translated using Weblate

Translated using Weblate (Romanian)
Currently translated at 82.1% (1561 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ro/

Translated using Weblate

Translated using Weblate (Portuguese (Portugal))
Currently translated at 81.3% (1546 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/pt_PT/

Translated using Weblate

Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (1901 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/pt_BR/

Translated using Weblate

Translated using Weblate (Portuguese (Brazil))
Currently translated at 100.0% (1901 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/pt_BR/

Translated using Weblate

Translated using Weblate (Polish)
Currently translated at 79.3% (1508 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/pl/

Translated using Weblate

Translated using Weblate (Norwegian Nynorsk)
Currently translated at 68.2% (1298 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/nn/

Translated using Weblate

Translated using Weblate (Dutch)
Currently translated at 82.7% (1574 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/nl/

Translated using Weblate

Translated using Weblate (Lithuanian)
Currently translated at 68.2% (1298 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/lt/

Translated using Weblate

Translated using Weblate (Korean)
Currently translated at 56.6% (1077 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ko/

Translated using Weblate

Translated using Weblate (Japanese)
Currently translated at 62.5% (1189 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ja/

Translated using Weblate

Translated using Weblate (Italian)
Currently translated at 100.0% (1901 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/it/

Translated using Weblate

Translated using Weblate (Italian)
Currently translated at 100.0% (1901 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/it/

Translated using Weblate

Translated using Weblate (Hungarian)
Currently translated at 86.9% (1652 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/hu/

Translated using Weblate

Translated using Weblate (Croatian)
Currently translated at 24.5% (467 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/hr/

Translated using Weblate

Translated using Weblate (Hebrew)
Currently translated at 44.5% (847 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/he/

Translated using Weblate

Translated using Weblate (Galician)
Currently translated at 86.7% (1649 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/gl/

Translated using Weblate

Translated using Weblate (French)
Currently translated at 97.1% (1847 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/fr/

Translated using Weblate

Translated using Weblate (Finnish)
Currently translated at 81.5% (1551 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/fi/

Translated using Weblate

Translated using Weblate (Basque)
Currently translated at 81.5% (1551 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/eu/

Translated using Weblate

Translated using Weblate (Estonian)
Currently translated at 80.4% (1530 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/et/

Translated using Weblate

Translated using Weblate (Spanish)
Currently translated at 96.8% (1842 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/es/

Translated using Weblate

Translated using Weblate (Greek)
Currently translated at 71.7% (1364 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/el/

Translated using Weblate

Translated using Weblate (German)
Currently translated at 88.0% (1673 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/de/

Translated using Weblate

Translated using Weblate (Danish)
Currently translated at 24.1% (460 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/da/

Translated using Weblate

Translated using Weblate (Czech)
Currently translated at 65.1% (1239 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/cs/

Translated using Weblate

Translated using Weblate (Czech)
Currently translated at 65.1% (1239 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/cs/

Translated using Weblate

Translated using Weblate (Catalan)
Currently translated at 83.0% (1578 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ca/

Translated using Weblate

Translated using Weblate (Bulgarian)
Currently translated at 13.5% (257 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/bg/

Translated using Weblate

Translated using Weblate (Asturian)
Currently translated at 75.8% (1442 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ast/

Translated using Weblate

Translated using Weblate (Arabic)
Currently translated at 15.9% (304 of 1901 strings)
Translation: aMule/Application
Translate-URL: https://hosted.weblate.org/projects/amule/amule-application/ar/

Co-authored-by: Diego Heras <[email protected]>
Co-authored-by: Got3nks <[email protected]>
Co-authored-by: Hosted Weblate user 54392 <[email protected]>
Co-authored-by: Marcelo Roberto Jimenez <[email protected]>
Co-authored-by: mslr8 <[email protected]>
Co-authored-by: தமிழ்நேரம் <[email protected]>
Co-authored-by: ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants