EC: detect dead amuled cleanly across all clients (#757) - #758

Summary

Refs #757. When amuled's EC connection died (process killed, network blip, FIN/RST lost) every client and the daemon itself failed to react usefully:

• amulegui — kernel observed FIN, asio reactor fired HandleRead with EOF, but dispatch landed in the empty CLibSocket::OnLost(){} default instead of the CECSocket::OnLost override CRemoteConnect actually overrides to flip the UI. The wedge symptom in the reporter's trace.
• amuleweb — sync EC mode means no async_read pending after auth, so EOF was never seen at the asio layer. HTTP requests continued to return template-shell HTML 200s with stale data — no error visible to the user, no exit.
• amulecmd — same root cause as amuleweb. Every command after disconnect failed but the loop kept prompting.
• amuled server side — when a remote-GUI / web / cmd client died ungracefully, the accepted CECServerSocket sat on the half-open connection for the default ~2h TCP retransmit timeout, holding m_ec_notifier references and per-client state. Heavy sharesets (Stoatwblr's seedbox class) accumulate these.

Three layered fixes

TCP keepalive on both ends (10103389a)

SO_KEEPALIVE + per-socket TCP_KEEPIDLE (30s) / TCP_KEEPINTVL (10s) / TCP_KEEPCNT (3) on every EC socket on both ends. The win is asymmetric and worth stating honestly: the primary benefit is amuled-side cleanup. amulegui sends INC_UPDATE requests every few seconds so its connection is rarely idle long enough for the 30s timer to fire — its detection comes from TCP retransmit timeout when amuled stops responding. amuleweb / amulecmd CAN sit idle long enough for keepalive to fire kernel-side, but app-level detection still needs a read/write attempt. Where keepalive really pays off is the amuled side: when a remote client dies ungracefully, the accepted CECServerSocket gets freed in ~60s instead of ~2h, so server-side resources don't accumulate.

OnLost dispatch fix for the async EC client (48a98504a)

CECMuleSocket multi-inherits from CECSocket and CLibSocket. Both declared virtual void OnLost(). With identical signatures they live in separate vtables in the combined object — there was no shared override, and the asio reactor's socket->OnLost() dispatch (through the CLibSocket vtable) landed in the empty base instead of the EC-layer override. Fixed by renaming the CLibSocket-side hook to OnLost(int) (mirroring the existing OnConnect(int) pattern) so CECMuleSocket can override it unambiguously and forward to the EC-layer virtual via a static_cast<CECSocket*>. This is the load-bearing fix for amulegui — without it, no application-level reaction fires even when the kernel sees the FIN.

Sync clients fire OnLost from ReadSync / WriteSync (766bd6ffa)

Sync mode has no pending async_read so the EOF that fires HandleRead → PostLostEvent for async clients never happens. ReadSync / WriteSync now call a new DispatchSyncLost helper on non-zero error_code — direct synchronous dispatch through the same wrapper->OnLost(0) path the async reactor uses, just from the calling thread since it's already in sync-mode context. CRemoteConnect::OnLost's NULL-m_notifier branch then prints "External Connection lost — exiting" to stderr and calls _exit(1) so the supervisor (systemd / docker / shell loop) handles restart and reconnect rather than the process serving stale data in limp mode.

Direct dispatch rather than PostLostEvent + wxQueueEvent because amulecmd's main thread is in fgets reading stdin, not the wx event loop — queued events would never be processed. amuleweb has a wxApp::OnRun main loop but it's not running while an HTTP request handler is on the stack. Direct dispatch from the sync-mode call site avoids both questions.

_exit instead of exit() because we're calling from contexts with asio worker threads about to be torn down; racing static destructors against them risks the kind of UAF #748 was about. OS reaps fds at process exit anyway.

Live verification on Linux VM

• amulegui + amuled killed with SIGKILL (FIN propagates): "Connection failure" dialog fires within ~1s.
• amulegui + iptables drop on amuled's outbound EC port (true half-open, no FIN): "Connection failure" dialog within ~33s. Detection came from TCP retransmit timeout (amulegui's periodic requests stopped getting ACKs), not keepalive — but the dispatch chain handled it correctly either way.
• amuleweb: first HTTP request after amuled killed triggers ReadSync EOF → OnLost → _exit(1). Subsequent requests get connection refused.
• amulecmd: first command after amuled killed prints External Connection lost — exiting. and exits with code 1.
• amuled server side: ss -to confirms SO_KEEPALIVE + TCP_KEEPIDLE=30 on every accepted EC socket.
• Builds clean on macOS and Linux.

Test plan

• amulegui graceful disconnect — UI dialog fires
• amulegui network-partition (half-open) — UI dialog fires via TCP retransmit
• amuleweb first failing request — clean _exit
• amulecmd first failing command — clean _exit
• amuled accept-side keepalive timer visible in ss -to
• Builds clean on macOS (Mac local) and Linux (Ubuntu ARM VM)