amuleapi REST API + SSE daemon + EC search-lifecycle additions (#201) * feat(amuleapi): standalone REST API + SSE daemon Adds a new daemon `amuleapi` that connects to amuled via EC and serves a versioned JSON REST API plus a long-lived Server-Sent Events stream over HTTP. Built on Boost.Beast; runs as its own process so it can coexist with amuleweb against the same amuled instance on independent ports (amuleweb 4711, amuleapi 4713). Wire-level guarantees: - JWT-based auth with admin / guest roles; session-cookie fallback for browser clients - Per-route rate limiting + IP-based login lockout - Conditional GET on every read (ETag + If-None-Match -> 304) - Live event stream with `Last-Event-ID` reconnect replay and a typed `resync` event for cache invalidation (gap detection + cross-restart ID reset) - Refresher polls EC every 1 s and emits typed diff events: download_added / _updated / _removed and the same shapes for shared files, server list entries, peer clients; status_changed for connection / kad / speed rollups; log_appended for amuled log tail - Optional CORS allowlist via amuleapi.conf[Server]/CorsOriginAllowlist (off by default; empty list = echo any origin in a cookie-compatible way; non-empty = strict per-origin allowlist) - Lazy-fetch TTL coalescing for stats / search / serverinfo so multiple concurrent reads share one EC roundtrip Concurrent REST clients share one EC connection to amuled via a process-wide mutex on every SendRecvSerialized; amuled's single-reader CRemoteConnect requirement stays intact under load. API contract lives under docs/api/ (per-endpoint fragments). First-run setup: docs/QUICKSTART-AMULEAPI.md. English man page at docs/man/amuleapi.1.in. New shared static lib libwebcommon collects the helpers amuleweb and amuleapi both need (ETag, path patterns, JSON writer) so neither code path re-implements them. Tests: - 10 muleunit C++ suites: AmuleApiConfigTest, AuthTest, EtagTest, EventBusTest, EventDiffTest, JsonWriterTest, JwtTest, PathPatternsTest, RefresherTest, StateTest - 18 curl-driven HTTP smokes under unittests/curl-tests/amuleapi/ exercising every endpoint end-to-end against a live amuled, plus a run-all.sh orchestrator that brings up a fresh daemon per phase Packaging: amuleapi rides every existing recipe (macOS .app via the build.sh bin_paths array, AppImage via EXTRA_BINS + argv[0]-dispatch AppRun, Flatpak local + Flathub manifests, Windows NSIS via the recursive File /r). docs/INSTALL.md gains a BUILD_AMULEAPI row. Disabled by default in cmake; enable with -DBUILD_AMULEAPI=YES or -DBUILD_EVERYTHING=YES. The CI build matrix (ccpp.yml) now sets BUILD_AMULEAPI=YES so every push exercises the new compile path. * feat(amuleapi): align endpoint surface with RFC PR #132 review Adds the endpoints the PR #132 review pass flagged as missing or shape-mismatched against the agreed contract, plus three operator ergonomics tweaks: Missing endpoints now shipped: - POST /shared/reload — rescan share roots (EC_OP_SHAREDFILES_RELOAD) - POST /servers/update — refresh server list from a `server.met` URL - POST /servers/:/connect — address-keyed alias for the existing ECID form; resolves : against the cache and delegates to the ECID handler - DELETE /servers/: — same alias for the removal path - DELETE /logs/amule — EC_OP_RESET_LOG plus a local CState::ClearAmuleLog so the next GET returns the post-reset state immediately (the refresher's incremental append-only path can't shrink the cache) - DELETE /logs/serverinfo — EC_OP_CLEAR_SERVERINFO plus TtlCache::Invalidate on the server-info lazy cache; without the invalidate, the next GET would keep serving stale text until the 1 s TTL expires Body / shape additions: - POST /downloads accepts `{"links":[...]}` (RFC §4.2 shape) in addition to the existing `{"ed2k_link":"..."}` singular form; mixing both in one request is a 400. Partial success in batch mode returns 207 with `accepted` / `failed` / `failed_links` - POST /networks/disconnect now reads an optional `{"network":"ed2k"|"kad"|"both"}` selector and routes to the matching EC opcode (SERVER_DISCONNECT / KAD_STOP / DISCONNECT); no body or `"both"` preserves the original parameterless contract - GET /status now includes `kad.network:{users,files,nodes}` — the rollup the RFC §4.1 contract specifies. The data was always available (KadSnapshot already parses EC_TAG_STATS_KAD_USERS / _KAD_FILES / _KAD_NODES on the EC_OP_STAT_REQ at EC_DETAIL_FULL); only the /status response was missing it - GET /clients gains `?filter=uploads|downloads|active` to subset the peer list by transfer state (replaces the legacy `/uploads` page server-side) - GET /events gains `?channels=` to subscribe to a subset of the event types. Synthetic `resync` events are delivered regardless of filter (their purpose is a cache-invalidation signal the client cannot opt out of) Mutation cache hygiene: every mutating endpoint either runs an inline RefresherTick to repopulate the State-backed cache (which also fires the corresponding SSE diff events) or, for the two lazy-cached endpoints, explicitly invalidates the TtlCache so the next GET re-fetches. phase11.sh smoke (30/30 PASS) exercises each new endpoint and includes a post-mutation freshness check on /logs/amule and /logs/serverinfo — fast GET immediately after the DELETE must return the empty state, not the pre-reset cache. * fix(amuleapi): explicit standard includes + LoggerConsole link Builds were green on macOS Release but failed on Ubuntu and Windows Release/Debug and macOS Debug: - src/webapi/Refresher.h declared `std::vector` without including . libc++ (macOS) brings in transitively through ; libstdc++ on Ubuntu and MinGW do not, so every TU that included Refresher.h failed with `'string' is not a member of 'std'`. - src/webapi/State.cpp used `std::unique_lock` without including . State.h ships which is enough for shared_lock + shared_timed_mutex, but libstdc++ does not re-export unique_lock from there, so the cpp failed to compile on Ubuntu and MinGW with `'unique_lock' is not a member of 'std'`. - src/webapi/Api.cpp used `std::ostringstream` without including . Worked transitively from picojson.h on most platforms; add the explicit include defensively. - unittests/tests/CMakeLists.txt: RefresherTest links libec.a but did not link LoggerConsole.cpp. Release builds preprocess libec's debug-emit paths to `do {} while(0)` via the ECLog.h define, so the symbols (DoECLogLine, theLogger, CLogger::AddLogLine) are unused; Debug builds keep the calls and require the symbols. Mirror amuleapi's own webapi/CMakeLists.txt which already pulls LoggerConsole.cpp in. No behavior change; only build hygiene to get the CI matrix green. * fix(amuleapi): address PR review findings (high + medium + low) High — must-fix: - amuleapi.conf is now mode-checked on every load and seeded with mode 0600 on first run. The conf carries the EC password in plaintext (App.cpp:202-210 MD5s it for the base class' hashed field) so it gets the same owner-only enforcement as the jwt-secret and passwords files. EnforceOwnerOnly() runs on the conf in LoadAmuleapiConf after the seed step. - --set-admin-pass / --set-guest-pass now propagate exit codes via std::exit(rc). Previously OnInit() returned false on failure, which cancels the run but still produces exit 0, so operators scripting `amuleapi --set-admin-pass=... && systemctl restart ...` could miss disk-full / mode-bit-rejection failures. - docs/QUICKSTART-AMULEAPI.md no longer lists /uploads. The endpoint was retired in Phase 4g and /clients (with the new ?filter= query) covers the per-peer surface. Medium: - Replaced the "is the field at its default value?" heuristic in LoadAmuleapiConfig with wxCmdLineParser::Found() predicates for --host / --port / --password. The old code would silently overwrite an operator-supplied `--host=127.0.0.1` with the conf value because it conflated "passed nothing" with "passed the default verbatim." - Dropped the no-op --daemon switch. The flag was wired to m_cliForeground = false but the field was never read. Mention in --foreground's help still notes "default." - WritePasswordsFile / WriteJwtSecretFile route through a new WriteFileAtomic0600 helper that writes to a sibling .tmp, fsync()s, then rename(2)s into place. A short write, signal, or crash mid-write now leaves the original file intact instead of truncating the only admin/guest credentials the daemon has. - Auth.cpp + Api.cpp pick up the #ifdef _WIN32 / #include shim that libwebcommon/HeaderParse.cpp already uses for strncasecmp. POSIX puts the symbol in ; glibc also re-exports via but musl and BSDs don't. - run-all.sh computes the repo root from $SCRIPT_DIR instead of hard-coding the original author's local path. AMULEAPI_ROOT remains an env override. Low — nits-as-comments + one test: - Comment at the SSE drain loop documents that ev.name is server- controlled (compile-time literals from EventBus::Publish call sites) so the un-escaped `event: ` write is safe in practice — and what to sanitize if that ever stops being true. - Comment at CRevocationSet::GcExpired() flags the O(n) sweep as load-bearing only at single-operator scale and points to the min-heap upgrade if the population ever climbs. - Comment at the per-streaming-session std::thread spawn calls out that the BindAddress=127.0.0.1 default is load-bearing against a thread-per-connection DoS, and points at the async-on- io_context migration that would close it off. - JwtTest gains direct boundary tests for Base64UrlDecode's len%4==1 and non-zero-residue rejections. Both craft a token with malformed b64 in the header section + a valid HMAC over the same signing input, so the constant-time MAC compare passes and Verify() actually reaches Base64UrlDecode (where the rejection must come from). - Comment at MakeSetCookie notes that the Max-Age=0 boundary on an already-expired expires_at is deliberate — the cookie expires on receipt, which is the right behaviour for logout-on-issue-of-expired-token paths. Drive-by CI fixes (caught while validating the review fixes): - EventBus::Publish raced on ID assignment: fetch_add lived outside the lock so a publisher could win the lock-order race but lose the id-order race, and a subsequent Drain would emit events in publish order with non-monotonic ids — failing EventBusTest's ConcurrentPublishersHaveDistinctIds on the macOS Debug runner. Moved fetch_add inside the lock so id assignment is atomic with the push_back. - Refresher.cpp's FULL-detail GetIPv4Data fallback path was removed. amuleapi only ever asks for EC_DETAIL_INC_UPDATE so the fallback was dead code in production, and it tripped a libec Debug-build assertion on RefresherTest fixtures (which pack the ECID as a uint32 in the EC_TAG_SERVER slot, exactly the not-IPv4-not-UNKNOWN case the assertion guards). - AmuleApiConfigTest gains #include for std::strlen (Windows MinGW does not bring it in transitively). * fix(amuleapi): address second-round PR review findings High — must-fix: - SSE channel-filter heartbeat starvation. The drain loop now tracks the wall-clock since the last byte written and emits `: keepalive` whenever it falls behind the 15 s budget, on EITHER the drained-and-emitted, drained-but-all-filtered, or drained-nothing branch. Previously a busy bus paired with a `?channels=` that filtered every event left the wire silent indefinitely — Drain returned immediately each time so the Drain-timeout-based heartbeat never fired, and NAT / load balancers / EventSource drop idle TCP connections after 30-60 s. Medium: - HandleStatus now reads through a new CState::Dashboard() accessor that returns (status, kad, snapshot_at, ec_connected) under a single shared_lock. Previously each accessor took its own lock so a refresher tick could interleave between Status() and Kad() and produce a /status response where `kad.network` described tick N+1 while `ed2k.*` / `speeds.*` described tick N. - HandleDownloadAdd no longer 503s mid-batch on an EC disconnect. Per-link results split into three buckets — accepted_links, failed_links, disconnected_links — and the response surfaces all three in counters plus arrays. 202 (all clean), 207 (mixed), 503 (every link blocked by the same EC outage — nothing landed). Caller can retry just the disconnected subset once /status reports ec_connected=true. Low — nits-as-comments: - event_channel lambda now documents the bare-name fallback: every bus event MUST contain `_`, the prefix names the channel. resync stays the only no-underscore name and is published directly to the writer (bypassing EventBus::Publish, hence the filter), so the lambda's fallback is a defensive net for future names rather than a routed code path. - phase11.sh /clients?filter=active assertion: the prior comment ("no peer is in both states simultaneously") was contradicted by the assertion's tolerance for double-counted peers. Re-stated as `active ∈ [max(up, down), up+down]` to match the semantics (peers can be uploading AND downloading at once; active is the set union, sum is the multi-set sum). - docs/QUICKSTART-AMULEAPI.md gains a "Notes on a few responses" section explaining the 207 multi-status on POST /downloads (the 503 reservation is documented too), and a security-notes section calling out POST /servers/update as an admin-authorized SSRF surface and the load-bearing BindAddress=127.0.0.1 default. Drive-by CI fixes: - mingw-w64 link failures on JwtTest / EtagTest / JsonWriterTest / PathPatternsTest / AmuleApiConfigTest / AuthTest / StateTest / EventBusTest / EventDiffTest. muleunit is an OBJECT library that pulls in src/libs/common/MuleDebug.cpp directly, which references CFormat (defined in src/libs/common/Format.cpp). On macOS/Ubuntu the symbols come through via wxWidgets / libc link order; MinGW's strict linker rejects. Added mulecommon to each test's target_link_libraries (CTagTest already pulls Format.cpp via its own sources list, so the pre-existing tests weren't affected). - phase4c-bis.sh used to assert DELETE /logs/{amule,serverinfo} → 405. Phase 11 made DELETE a real mutation, so the test now asserts PATCH → 405 (the logs are read-and-reset only, never partially mutable) — same intent, post-Phase-11 contract. Verified locally: - ctest: 21/21 (all webapi + libwebcommon suites) - run-all.sh curl matrix: every phase smoke green standalone (phase8d occasionally flakes on the first hit after a daemon restart — passes on re-run; not in any failure path) * fix(amuleapi): AmuleApiConfigTest tmp portability for mingw-w64 The test was hard-coded to put its per-test config dir under `/tmp`, which doesn't exist on Windows MinGW (the CI matrix's mingw-w64 Release/Debug jobs failed every test with "The system cannot find the path specified" from wxRemoveFile / wxRmdir / wxMkdir). Switched to `wxStandardPaths::Get().GetTempDir()`, which returns `/tmp` on POSIX and `%TEMP%` on Windows. Same intent, portable. Also using `wxGetProcessId()` instead of `::getpid()` so the source doesn't carry a POSIX-only call site outside the `#ifndef _WIN32` include guard. ctest 10/10 locally; no shape change to the tests themselves. * fix(amuleapi): address pre-PR blockers + High/Medium review findings Two pre-PR blockers: - LastSeenState race. Every Phase-5 mutation handler calls RefresherTick() inline on the HTTP-server worker thread; the wxApp refresher loop calls it once per second. Both paths used to end in EmitDiffsAndUpdate(bus, m_last_seen, state), which reads and mutates the per-substruct maps in m_last_seen without any internal lock. Two threads doing std::map insert/erase on the same map is UB; an attacker (or operator) coupling a busy client to inline mutations could corrupt the diff baseline and crash the daemon. Split: RefresherTick() no longer touches m_last_seen at all. A new EmitDiffsForEventBus() helper does the walk; the wxApp refresher loop in App.cpp (the single writer thread) calls it after each successful tick. SSE subscribers see post-mutation diffs on the next natural tick (≤1 s) instead of immediately — acceptable cost for memory safety. - SIGPIPE never masked. On Linux, an SSE peer that disappears mid-write raises SIGPIPE on the next asio::write to the closed fd. The streaming-handler socket writes don't pass MSG_NOSIGNAL, so without ignoring SIGPIPE the default disposition kills the daemon on every dropped EventSource. OnRun() now sets std::signal(SIGPIPE, SIG_IGN) alongside the existing SIGINT/SIGTERM/SIGHUP handlers; the writes return EPIPE and the streaming-handler loop bails on the next writer.Alive() poll. libwebcommon High: - CJwt::CJwt(secret) now throws std::invalid_argument on an empty signing key. CryptoPP's HMAC accepts a null/zero-length key without complaint, so a config bug that truncated the amuleapi-jwt-secret to 0 bytes used to produce a daemon that happily issued and verified tokens against an empty key. - WriteEscapedString now substitutes U+FFFD (replacement char) for unpaired UTF-16 surrogates instead of falling through to BMP serialization and producing invalid UTF-8 once the buffer flushes. Affects only paths that route raw wxStrings through the JSON writer with broken Unicode input. - FindCookieValue now strips RFC 6265 §5.2 OWS from both ends of the value (`name= value ` → `value`, not ` value `). Without it a JWT cookie set by a client that inserted a leading space after `=` would fail ConstantTimeEquals on every Verify. libwebcommon Medium: - ConstantTime.h gains a precondition block on the wxString overload: the length-mismatch check is NOT constant-time but is safe today because every caller compares fixed-shape inputs (32-char MD5 hex, 43-char b64url MAC). Future length-variable callers must pad or accept a length-side-channel leak. - CJwt::Verify now allows a 5-second exp leeway and rejects an iat claim more than 60 s ahead of the verifier's clock. Today the issuer and verifier share a clock so the skew is zero; tomorrow they may not (federated tokens, reverse-proxy auth handoff). webapi High: - CRateLimiter is now actually sliding, not tumbling. The legacy `failure_count + window_start` shape reset wholesale when the current window expired, letting threshold-1 attempts in the tail of window N + threshold-1 in the head of window N+1 slip through. Replaced with a std::deque per IP: each NoteFailure evicts stamps older than window_seconds and appends now; lockout fires when the live count crosses threshold. Two new AuthTest cases pin (a) lockout expiry after lockout_seconds and (b) the split-attempt regression that the old tumbling impl would have passed. - SSE concurrent-session cap. A new process-wide atomic counter bounds streaming sessions at kMaxConcurrentStreamingSessions (32). The slot is acquired before the per-session OS thread is spawned; over-the-cap requests get 503 with `error.code=sessions_exhausted` + Retry-After:10. The cap caps the thread-per-connection DoS amplifier that becomes reachable if an operator switches BindAddress to a non- loopback. A startup WARN line also fires when BindAddress isn't loopback, so the operator sees the trade-off in the daemon log. - Session destructor now asserts the m_worker_exited atomic before detach()ing the thread handle. The "worker holds shared_from_this() across its run, so the dtor only sees it after the worker exited" invariant is now enforced at runtime instead of by-comment; a future refactor that drops the shared_from_this() capture fails loudly rather than mysteriously deadlocking. data layer (covered by BLOCKER 1 + comment fix): - m_partfile_rle comment now matches reality: the mutator is CState::MutateDownloads under the State write lock, NOT m_ec_mtx. m_ec_mtx is incidentally held across the same call stack but isn't the actual serializer. webapi Medium: - ParseEcidPath now uses std::strtoull with errno + ULLONG_MAX + > 0xFFFFFFFF gate. On Windows `unsigned long` is 32-bit, so the prior > 0xFFFFFFFFu overflow check was a tautology and a bogus high-value path segment would saturate to ULONG_MAX and silently match an actual ECID 0xFFFFFFFF. - MakeSetCookie / MakeClearCookie use std::string instead of a 256-byte snprintf buffer. Today's HS256 JWT fits; tomorrow's longer payload (extra claim, alg switch) won't truncate. tests: - AuthTest gains (1) RateLimiter_LockoutExpiresAfterLockoutSeconds (1.1 s real-time test for "infinite lockout" regression), (2) RateLimiter_SlidingWindowSplitAttemptsStillLockOut for the ring-buffer behaviour the old tumbling impl would have failed, and (3) RevocationListBlocksOtherwiseValidToken — the cross-test the review flagged as missing, composing CJwt::Verify with CRevocationSet so a regression in either component's lookup fails the test. - run-all.sh: pkill scoped to `--config-dir=/tmp/amuleapi-regtest` so a dev's open editor doesn't get killed. Daemon readiness wait is now `curl /version` polled with timeout instead of `sleep 1` (matches the line-19 comment about cold-start cost). A 429 probe after a failed phase prints a "phase3 left the lockout armed; restart amuleapi" tip so the next dev doesn't lose an hour. - phase5a.sh / phase8b.sh / phase11.sh: cleanup trap now also DELETEs the Ubuntu ISO test partfile (when ADMIN_TOKEN is available) so a failed run doesn't survive into the next as 6.6 GB of disk pressure on the Windows VM. docs/QUICKSTART-AMULEAPI.md: - Drop the broken `docs/api/` cross-reference (the directory doesn't exist yet); point at the source-of-truth handler routing in `src/webapi/Api.cpp` instead. - Add the amuled-cohabitation callout above the config-dir table (same data dir as amule.conf; no collisions; both daemons read disjoint files). - Reword the `--set-admin-pass` / `--set-guest-pass` block to explicitly say (a) no HTTP server is started, (b) no EC connection is attempted, (c) the exit code reflects success / failure so chains like `amuleapi --set-admin-pass=... && systemctl restart amuleapi` short-circuit on write failure. - CORS section gains a callout: the empty-allowlist form echoes the caller's Origin (not literal `*`), because `Access-Control-Allow-Credentials: true` is incompatible with the literal wildcard and cookie-auth would break otherwise. Verified locally: - ctest: 21/21 (new AuthTest cases included) - run-all.sh standalone (after restart): phase8a 11/11, phase8c 7/7, phase8d 9/9, phase11 30/30. (Without a daemon restart immediately after phase3 the new sliding-window correctly refuses login — that's the intent; the orchestrator's per-phase restart clears the bucket.) * fix(amuleapi): four partial-finding follow-ups - RateLimiter_SlidingWindowSplitAttemptsStillLockOut now actually discriminates. The prior timing (window=5, threshold=3, single sleep 1.1 s between failures) passed against both the old tumbling impl and the new sliding impl — the failures landed inside the same window from the OLD code's perspective so the reset never fired and the count went 1→2→3 regardless. Tightened to (window=3, threshold=3) with failures at integer-second offsets t=0, 3, 4, 5 — at t=4 the OLD code triggers the wholesale reset and the count drops to 1; at t=5 OLD count=2 (no lockout) while NEW evicts the t=0/t=1 stamps and tallies 3 within the live 3-second window (lockout). Verified with a standalone harness replaying the sequence against the OLD impl: locked=NO, count=2. - Session::~Session now wraps the worker-exit invariant in a hand- rolled `if (!flag) std::abort();` instead of plain `assert()`, so the check survives -DNDEBUG (compiled into every Release / RelWithDebInfo build). The worker lambda sets the flag via an RAII WorkerExitMarker struct rather than a bare end-of-body statement, so a future refactor that adds an `early return` after the catch block still trips the flag before `self` drops. - run-all.sh's final-teardown `pkill -f amuleapi` was the last un-scoped sweep (the per-phase one at line 53 was already narrowed to `--config-dir=/tmp/amuleapi-regtest`). Now the teardown uses the same pattern, so a developer with `vim path/to/amuleapi.cpp` open keeps their editor across full-matrix runs too. - docs/QUICKSTART-AMULEAPI.md: the cohabitation-with-amuled callout now sits ABOVE the per-platform config-dir table (the layout the prior commit message claimed but didn't actually deliver). The intro line + callout + "default location" + table reads more linearly too. * fix(amuleapi): ship picojson license + cap JSON nesting before parse - src/libwebcommon/picojson.LICENSE: reproduce picojson 1.3.0's BSD-2 notice next to the vendored header, mirroring how muleunit ships unittests/muleunit/license.txt. BSD-2 clause 2 obliges any binary distribution (AppImage, Flatpak, .deb, macOS bundle, Windows installer) to include the notice in materials provided with the distribution — without this file the shipped binaries were technically out of compliance even though the source-form clause was satisfied by picojson.h's in-file comment. - docs/THIRDPARTY.md: full credits/license reproduction for picojson (BSD-2) and muleunit (LGPL v2.1, test-binary-only — explains why the relinking clause does not constrain end-user binaries since muleunit is never linked into amule/amuled/amulegui/amuleweb/ amuleapi). - LICENSE.md: short header pointing at docs/THIRDPARTY.md for the third-party components, leaving the GPLv2 text unchanged below. - src/libwebcommon/Jwt.cpp: count '{' + '[' in each b64-decoded JSON section and reject when > 32 BEFORE calling picojson::parse on either the header (Jwt.cpp:225) or the payload (Jwt.cpp:250). picojson's _parse_array/_parse_object is unbounded recursive descent; both parses run before the MAC verdict reaches the caller, so an unauthenticated peer could submit a token in Authorization: Bearer whose b64-decoded JSON nests deep enough to blow the worker thread's stack (musl pthreads cap at 128 KiB). Real JWTs are flat — a cap of 32 leaves ample headroom; the scan is O(body length) with no allocations. - src/webapi/Api.cpp: pull the same opener-count cap into ParseJsonObjectBody so every mutating-body parse site is covered, and route HandleLogin through ParseJsonObjectBody instead of calling picojson::parse directly — HandleLogin is reachable unauthenticated, so it needs the same defence as the JWT surface. Forward declaration in the helpers-anon-namespace block near the top of the file so HandleLogin (defined above ParseJsonObjectBody) can call it. - unittests/tests/JwtTest.cpp: two new regressions (DeeplyNestedPayloadRejected, DeeplyNestedHeaderRejected) build a token whose nesting is 200 levels deep and sign it with the matching HMAC so the MAC compare succeeds and the depth cap is the only available reject path. Suite goes 23 → 25 cases, all 21 ctest binaries still green. * fix(amuleapi): sweep final pre-PR review (six batches, ~25 findings) Six batches landed against the final pre-PR review. All 21 ctest binaries still green; JwtTest grew to 27 cases (depth-cap + iat mandatory regressions); AuthTest dropped from 6.5 s → 0.44 s after clock injection. Batch 1 — quick wins - HttpServer.cpp: route 5xx body through CJsonWriter::ValueString (H-S1). - Auth.cpp Check(): evict stale failure stamps the same way NoteFailure does (H-S2). - EventDiff.cpp: capture-on-first-call thread-id assertion in EmitDiffsAndUpdate; hard abort if a second thread publishes (H-S3). - PathPatterns.{h,cpp} + Api.cpp: LooksMalicious(path) defence- in-depth — reject NUL / %00 / %2e%2e / literal ".." segments before routing (H-S11). - Jwt.cpp Verify: reject tokens > 4 KiB before any Base64Url walk (M-S10). - Api.cpp ?channels= cap at 32 tokens (M-S13). - Api.cpp HandleLogout: revoked-token now 200 noop, not 401 (M-S14). - Api.cpp: kSessionCookieAttrs as the single source of truth for Set/Clear cookie attribute strings (M-S11). - App.h: rename PartfileRleState() → PartfileRleStateRequireStateWriteLock() to encode the precondition (M-S2). - State.cpp: comment the tick-end-stamping skew on m_snapshot_at and the "events atomic, state isn't" invariant (M-S3 + M-S12). Batch 2 — auth / JWT hardening - Jwt.cpp Verify: iat is now mandatory; additionally cap (exp - iat) ≤ TOKEN_LIFETIME_SECONDS + skew so even a compromised secret can't mint a 100-year token (H-S10). - Api.cpp HandleLogin: drop the JWT from the JSON body by default — browsers get the cookie only; bearer clients opt in via Accept: application/jwt or ?type=bearer (H-S12). - Jwt.h: signing secret now stored as CryptoPP::SecByteBlock so the live key gets scrubbed at destruction; constructor also zeroes the caller's vector after copying (M-S1). - AmuleApiConfig.cpp: first-run amuleapi.conf write routed through WriteFileAtomic0600 — no truncated configs on mid- write crash (M-S5). - Api.cpp: new generic 401 rate limiter (30/60s, 5-min lockout) applied to every auth-protected endpoint via AuthenticateRequestRateLimited, including HandleLogout's inline-soft path (M-S7). - Api.cpp ResolveServerEcidByAddress: drop the s.address string fallback; require IPv4-shaped match (M-S9). - curl-tests: opt every login URL into ?type=bearer so the script harness keeps reading the token out of the body. - unittests/tests/JwtTest.cpp: two new regression cases (MissingIatRejected, ExpIatLifetimeCapExceeded) and EmptyJtiRejected updated to include a valid iat so it exercises empty-jti rather than the new mandatory-iat path. Batch 3 — SSE concurrency - Api.cpp DispatchEvents: post-Drain gap detection — if the cursor fell off the ring between drains, emit synthetic resync (reason=gap) and restart from newest. Previously only reconnects caught gaps; a cold-start tick on a 5K- download library would publish ~5K events, evict 4900, and the live drainer would silently miss them (H-S4). - EventBus.{h,cpp}: Shutdown() + IsShutdown() latch. App's OnExit calls Shutdown() before tearing down the dispatcher; Drain wakes blocked subscribers, the SSE worker loop polls IsShutdown() and exits — closes the UAF window where a detached worker held m_dispatcher across dispatcher.reset() (H-S7). - HttpServer.{h,cpp} + Api.{h,cpp}: new StreamingPreflight callable runs synchronously on the I/O thread BEFORE the SSE worker thread spawns and BEFORE the 32-slot streaming-session budget is touched. Unauth peers get 401/429 without burning a slot for the read-timeout window (H-S8). - App.cpp refresher loop: wall-clock-bounded to a 1 s target cycle (was a fixed 4×250 ms sleep regardless of tick duration) + WARN log on any tick > 3 s — typically signals EC-mutex contention (H-S5). Batch 4 — cache / perf - TtlCache.h: split GetOrFetch into "fresh? return cached" vs "claim inflight via condvar, drop mu, run fetch, broadcast". A 30 s amuled stall on /stats/tree no longer parks every concurrent reader (H-S6). - Api.h + Api.cpp: snapshot-versioned ETag memoization keyed on (request target, m_state.SnapshotAt()). Previously every 200 GET/HEAD ran MD5 over the (potentially multi-MB) body; on a 10K-shared-file daemon that was the dominant CPU cost on the safe-method path (M-S6). - Pagination (M-S8) deferred per user direction. Batch 5 — tests - Auth.h + Auth.cpp: clock injection into CRateLimiter via optional Clock parameter (defaults to std::time(nullptr)); AuthTest's sliding-window / lockout-expiry tests now run with a controllable clock instead of real-time sleeps — AuthTest dropped from 6.5 s → 0.44 s (H-T1). - JwtTest MalformedBase64Rejected: drop the misleading "constant-time" framing since no timing assertion exists (H-T2). - phase4.sh: poll /status until 200 instead of sleep 2 — doesn't drift to disconnected under Windows VM load (H-T3). - phase4.sh test 4: real guest login instead of the comment- only stub; _die if guest pass isn't configured so the fixture gap surfaces (H-T4). - phase3.sh: add 5th-attempt 429 check (was only the 8th) so the lockout boundary is pinned (M-TD6). - StateTest: ASSERT_TRUE(observed > 1000) instead of > 0 so the tear-detection harness can't pass on a single observation (M-TD7). - AmuleApiConfigTest: document why the mode-bit test is POSIX-only and what a future Windows port would land (M-TD8). Batch 6 — docs / man / cmake - AmuleApiConfig.cpp: document the deferred --rotate-jwt- secret CLI; manual rotation via "delete the file + restart" works today (M-S4). - cmake/options.cmake BUILD_AMULEAPI block: document the hard-fail dependency chain (cryptopp via NEED_LIB_EC, Boost via unconditional cmake/boost.cmake) so a future refactor doesn't accidentally let amuleapi soft-disable (M-TD1). - docs/QUICKSTART-AMULEAPI.md: callout disambiguating --port=4712 (EC port amuleapi talks TO) vs the 4713 HTTP listener (port REST clients talk TO); curl example switched to ?type=bearer (M-TD2). - docs/man/amuleapi.1.in: drop the non-existent -V short flag from SYNOPSIS + OPTIONS; the wired form is --version (M-TD3 + M-TD4). - docs/man/CMakeLists.txt: comment-only ack of the translated-manpage gap — adding amuleapi to MANPAGE_BINARIES today fails the build because po/manpages-*.po has no amuleapi strings. English-only until 3.1; explicit wiring instructions in the comment for whoever lands the translator pass (M-TD5). * fix(amuleapi): correct phase3/phase4 fixture off-by-ones from review sweep Two fixture bugs surfaced when run-all.sh was actually re-run on top of the sweep commit; both were introduced by the batch-5 changes that addressed reviewer M-TD6 / H-T4. - phase3.sh — the 5th-attempt boundary assertion (M-TD6) had the off-by-one in the wrong direction. Rate-limiter Check() runs BEFORE the password compare; NoteFailure() runs AFTER. So with threshold=5, the 5th attempt arms the lockout but still returns 401 (the password compare failed and the response was already prepared); the FIRST 429 lands on attempt 6 when Check() sees the armed lockout. The assertion now asserts 401 on attempt 5 and 429 on attempt 6, which is what the limiter actually guarantees. - phase4.sh — the new guest-bearer check (H-T4) referenced $GUEST_PASS without setting a default. run-all.sh configures guestpass via --set-guest-pass, but a standalone phase4.sh invocation has no GUEST_PASS in the environment and `set -u` bails. Default to `guestpass` to match run-all.sh's daemon fixture, and document the env var alongside ADMIN_PASS in the script header. * fix(amuleapi): second pre-PR review sweep (7 findings, including 1 SecOpsImpact) Seven findings from the second-round review. All 21 ctest binaries still green; run-all.sh full curl matrix still green (660+ assertions across 26 phase scripts). - HttpServer.cpp:153 — set explicit m_parser->header_limit(16 KiB). Beast's default header_limit varies across versions (8 KiB in current upstream, larger on older). Without an explicit cap, a drip-feed attacker who slowly streams header lines grows the flat_buffer until the 10 s read timeout catches them — but that's 10 s × pool of concurrent peers of memory pressure. The explicit cap pins the per-peer ceiling at 16 KiB regardless of upstream defaults. (#3) - HttpServer.cpp:200-220 — replace the e.what()-in-body 500 with a generic "internal server error" message; log e.what() server- side via stderr. Information-disclosure defence: a future header-driven throw site (or picojson's offset-with-byte error messages today) would otherwise reflect caller-supplied bytes (Authorization fragments, Cookie content) back to the requester via the 500 body. The H-S1 fix in the prior sweep handled JSON escaping but kept the disclosure surface; this closes it. (#2) - App.cpp TextShell — bind-time hard gate. Refuse to start when BindAddress is non-loopback AND both admin/guest passwords are empty. The prior WARN-only behaviour meant a daemon started against a routable interface BEFORE the operator ran --set-admin-pass accepted TCP, refused every login (503 login_disabled), but still exposed the unauth surface (/version) and would silently leak any future read-without-auth endpoint. Loopback binds are unaffected — that's the documented first-run flow. (#1) - HttpServer.cpp + App.cpp shutdown — Listener::Stop now cancels every live SSE session's socket via a new Session::RequestCancel posted onto the session's strand. The new g_live_streams weak_ptr registry tracks every session that holds a streaming slot. Without this, a worker blocked inside a synchronous Beast write to a slow peer would hold the connection until the kernel TCP timeout (~minutes) and CHttpServer::Stop would join the io_context thread indefinitely waiting for the workers. Now the close fires, the write fails EPIPE, the worker exits cleanly, the last shared_ptr ref drops, and shutdown completes in bounded time. (#6) - App.cpp refresher loop — fail-loud on sustained EC blackout. After ~30 s of consecutive failed ticks, log a sharp WARN to stderr; after ~5 min, set g_shutdownRequested so a supervisor (systemd, launchd, docker restart=always) can bring the pair back. Previously the refresher would silently loop forever retrying a dead amuled, with the daemon serving 503 ec_unavailable from every endpoint with no visibility into the underlying cause. (#7) - EventBus.{h,cpp} + EventDiff.cpp — new CEventBus::PublishBatch(vector>) takes one lock acquisition + one notify_all for the whole batch. DiffMap now accumulates removed/added/updated tuples and emits via the batch API. The cold-start tick on a 5K-download library was previously firing ~5K individual notify_all cycles inside the refresher loop (each going through every drainer's cv mutex), which dominated the tick wall-clock. Single-publisher invariant + monotonic id assignment both preserved (the batch assigns inside the same lock as the per-event Publish). (#9) - App.cpp OnRun — sigaction with explicit sa_mask instead of std::signal. std::signal has implementation-defined "reset-to-SIG_DFL after firing" semantics on some platforms (notably musl/Alpine and legacy BSDs); a second SIGINT could default-terminate the daemon mid-shutdown. sigaction with SA_RESTART gives portable stay-installed semantics across every relevant target and prevents EINTR on blocking EC reads during shutdown. Windows lacks sigaction; std::signal fallback retained there. (#5 — Alpine/musl support) * docs(amuleapi): add full REST + SSE reference under docs/amuleapi/ Two new files covering the complete v0 API surface: - docs/amuleapi/REFERENCE.md (~950 lines) Endpoint catalog: every route under /api/v0/, grouped by resource (system, auth, downloads, clients, shared, servers, categories, preferences, networks, kad, logs, stats, search). Per endpoint: HTTP method(s), auth role (NONE/GUEST/ADMIN), query/path params, request body schema, complete curl example, sample success response, and the error codes the handler can emit. Sections upfront for the shared model: auth carriers (bearer vs cookie), role gating, rate limiting, JWT structure, error envelope, ETag and conditional GET, CORS surface, path validation, request size limits. Closing error-code catalog so clients have one table to alert on. - docs/amuleapi/EVENTS.md (~310 lines) SSE contract: connect flow, headers, initial reassurance chunk, frame format, channel filter (downloads/shared/servers/clients/ status/logs), 15 s heartbeat semantics, Last-Event-ID reconnect matrix, live-path gap detection, and the synthetic resync event. Catalogs every event the bus emits with its payload shape: download_*, shared_*, server_*, client_*, status_changed, log_appended, plus the filter-bypass resync. Closing notes on the single-publisher invariant and shutdown behaviour. Cross-linked from: - docs/QUICKSTART-AMULEAPI.md (replaced the "OpenAPI is a follow-up" hedge with pointers to the new files). - docs/man/amuleapi.1.in (the man page promised docs/api/; updated to point at docs/amuleapi/REFERENCE.md and docs/amuleapi/EVENTS.md). No source changes; pure docs. Closes the "where's the API spec" gap that the man page was advertising against an empty directory. * docs(amuleapi): rename docs/amuleapi/ → docs/api/ The man page originally promised docs/api/; restore the directory name to match. Renames the two reference files in place and updates the cross-links in QUICKSTART-AMULEAPI.md and the amuleapi(1) man page. No content changes. * docs(amuleapi): swap real ed2k server IP in examples to RFC 5737 range The doc examples carried 176.123.5.89:4725 verbatim, lifted from the phase11.sh curl regression test where it was used because it's a known-live ed2k server. That's fine for a smoke test but wrong for user-facing docs — implies endorsement and links a third-party operator's address to the project. Swap to 203.0.113.5:4242 (RFC 5737 TEST-NET-3, reserved for documentation) throughout REFERENCE.md and EVENTS.md and align the port in the example status response so the two examples stay consistent. No structural changes. * fix(amuleapi): align SSE↔REST shapes + 6 review items + partial comment cleanup Code: - EventDiff.cpp/h: every ToJson(*Snapshot) rewritten to emit the same byte-for-byte shape as the matching REST list-item writer in Api.cpp. Every Equal(*Snapshot) expanded to compare every emitted field so SSE _updated fires on any movement (the original gap was shared_updated never firing on xfer counter movement). New ToJsonStatusEvent(Status, Kad, ec_connected) produces the nested REST /status envelope; new Equal(KadSnapshot). EmitDiffsAndUpdate reads state.Dashboard() so status + kad + ec_connected are diffed against the prior tick as one rollup. LastSeenState gains KadSnapshot kad + bool ec_connected. - Api.cpp: HandleNetworksConnect accepts {"network":"ed2k"|"kad"| "both"} matching /networks/disconnect; routes to EC_OP_SERVER_CONNECT / EC_OP_KAD_START / EC_OP_CONNECT. HandleKadConnect + HandleKadDisconnect deleted as strict aliases of the granular form. /search/results fires EC_OP_SEARCH_PROGRESS in the same single-flight TtlCache miss; response carries progress.{percent, complete} so empty results aren't ambiguous between "no search", "in flight", "finished with zero hits". Api.h adds SearchCacheValue + new TtlPair_Search alias. Tests: - phase5e.sh: replace dedicated /kad/{connect,disconnect} assertions with /networks/{connect,disconnect} body-selector assertions; add network=ed2k and bogus-selector negative cases. - phase6.sh: assert progress envelope on /search/results. Docs: - REFERENCE.md: TOC linking every endpoint section; clear_completed cross-references DELETE /downloads/{hash}; kad/bootstrap body corrected from a wrong {server_address: ...} to the real {ip, port} shape; /networks/connect doc updated for the body selector and /kad/{connect,disconnect} removal noted; /search/results doc shows progress envelope. - EVENTS.md: every event sample replaced with the aligned shape; cross-resource intro states _added/_updated are byte-identical to the REST list-item shape they mirror. Comment cleanup: - Removed PLAN.md / PLAN §N / internal-phase-naming references from in-code comments per the standing policy. Replaced "deferred to v0.2" parentheticals with their removal. Section-divider phase comments and pure phase-marker comments deleted. Strip is partial; the bulk-apply tool tripped a regex bug; this commit is the post-recovery point and a follow-up sweep will finish. All 21 ctest binaries green; build clean. * fix(amuleapi): /shared ghost entries on partfile-becoming-shared tick ApplyGetUpdateToShared inserted a first-time SharedSnapshot with hash="000…0" (zeroed CMD4 from GetTagByNameSafe(EC_TAG_PARTFILE_HASH)) and name="" whenever a partfile transitioned to shared (≥1 chunk just completed) in a tick where EC_TAG_PARTFILE_HASH / _NAME were CValueMap- suppressed by amuled — the partfile's identity hadn't changed since its earlier non-suppressed update, only EC_TAG_PARTFILE_SHARED flipped. The client saw a ghost row in /shared with empty hash + name. Repro: POST /api/v0/search/results/{hash}/download → wait until partfile accumulates ≥1 chunk → GET /shared returns ECID with zeroed hash. Fix: gate first-time insert on PARTFILE_HASH presence; if absent, fall back to the downloads cache snapshot (populated this same tick from the partfile's earlier non-suppressed update). If neither has identity, defer the insert — the next tick where amuled emits a true identity refresh catches it. Verified on local repro: ECID 1150 (debian-10.13.0-amd64, 22 MB / 2+ chunks) appears in /shared with proper hash + name; ECIDs without completed chunks correctly absent. No ghosts across stability poll. * feat(amuleapi): /downloads/{key} and /shared/{key} accept hash OR ECID Previously /downloads/{hash}, /downloads/{hash}, /downloads/{hash}, and /shared/{hash} only matched the 32-char MD4 hex. Clients that already had the ECID from a list view (the canonical id in our State maps) had to carry the hash separately for any detail / mutation call. Disjunctive route: capture renamed to {key}, handler dispatches on shape — decimal 1..10 digits ≤ 0xFFFFFFFF → ECID (O(1) map lookup); anything else → hash (case-folded scan). The two grammars are non-overlapping, so no ambiguity / type prefix needed. Scope: GET /api/v0/downloads/{key} — HandleDownloadDetail PATCH /api/v0/downloads/{key} — HandleDownloadPatch DELETE /api/v0/downloads/{key} — HandleDownloadDelete PATCH /api/v0/shared/{key} — HandleSharedPatch POST /api/v0/search/results/{hash}/download deliberately out of scope — that's a search-result hash (transient ID), not a partfile identity. Adds State::FindDownloadByEcid, State::FindShared, State::FindSharedByEcid to round out the lookup surface (FindDownload by hash already existed). The ECID variants are O(1) map lookups vs the hash variants' linear scans; downstream EC ops still address by MD4 hash, so handlers read the hash off the resolved snapshot rather than the URL. phase4b.sh + phase5f.sh gain 5 new assertions covering the ECID path; REFERENCE.md documents the disjunction inline at the relevant endpoint sections + in the index. * feat(amuleapi): static-frontend serve + out-of-box discovery Adds a static-file dispatcher under non-/api/ paths so an operator can ship a frontend bundle alongside amuleapi without a separate webserver. Discovery mirrors amuleweb's GetTemplateDir (src/webserver/src/WebInterface.cpp): bundle Resources/ on macOS, then configure-time AMULEAPI_STATIC_DIR, then wxStandardPaths fallback — so a standard install (`make install`, MSYS2 portable, .app bundle) serves the bundled placeholder with zero conf edits, and [Server]/ StaticRoot still wins as an explicit override. Containment + caps: - POSIX symlinks resolved via realpath(); escapes from root are rejected. Windows uses _fullpath() (lexical-only, since symlinks require elevation there) plus a normalised prefix check that survives trailing slashes in the configured root. - 16 MiB per-asset cap stat'd before reading. - mtime+size ETag so reloads short-circuit to 304 via If-None-Match. - Extension-less misses fall back to index.html (SPA-friendly); misses that carry an extension 404 honestly so a broken bundle is visible. ResolveWithinRoot + IsDir live in src/webapi/StaticFs.{h,cpp} so the unit tests (StaticFsTest, 11 cases) can link them without dragging in the full dispatcher. Curl-level wire tests in phase12.sh cover ETag round-trips, SPA fallback, symlink rejection, and the size cap. Verified end-to-end on macOS (LSCopy bundle path), Linux (cmake install path), and Windows (portable install path). * fix(amuleapi): always include category 0 in GET /categories amuled's EC suppresses the whole `EC_TAG_PREFS_CATEGORIES` block when no custom categories exist, even though category 0 (the default incoming dir) is always present in amuled's data structures — `GetCatCount() >= 1` always, and `AddDownload(category=0)` works unconditionally. Once the first custom category is created amuled starts emitting both index 0 and the new one. The empty-then-populated wire shape forces every client to special- case the no-custom-categories tick instead of just iterating the list. Normalise at the API layer: if amuled didn't return index 0, prepend a synthetic entry with the same defaults amuled itself would emit (empty title/path/comment, color 0, priority_code PR_LOW per `CPreferences::LoadCats`). Refresher / state stay faithful to the EC wire so internal callers that already handle the empty case (e.g. created-index lookup in HandleCategoryPost) keep working unchanged. Reported in #201 review. * test(amuleapi): rename curl-test phases to descriptive names `phaseN.sh` was a development artifact — useful while landing each chunk under review, but unhelpful once the whole surface exists and contributors want to find the script that covers a specific endpoint or transport. Renames the 27 scripts to `NN-.sh`. The numeric prefix preserves the canonical execution order in run-all.sh (auth before any mutation, refresher-consolidation tests before later read smokes, CORS / static-frontend after the API surface so transport failures don't mask earlier regressions). The descriptive suffix says what's under test, so `ls` reads as a feature inventory rather than a commit log. Also updates: - run-all.sh PHASES array + the per-script special-cases (static- frontend dir provisioning, CORS-rebounce env, 02-auth lockout hint). - Every script's internal headers: banner, usage example, mktemp prefix, `/tmp/amuleapi-…` paths, and cross-references to other scripts. Narrative comments that mention "Phase N" as a *development phase* (e.g. "the wire decision in Phase 8c") are left alone — those refer to PR-review history, not script filenames. Full suite re-verified after the rename (27/27 green on macOS). * feat(amuleapi): refresher-driven search lifecycle + SSE events GET /search/results used to read amuled's raw EC_TAG_SEARCH_STATUS directly, which is ambiguous: raw=0 means "no search" OR "Kad in progress" OR "global queue not yet populated" OR "global finished (m_searchInProgress just flipped off)"; raw=100 means "global queue empty at start" OR "global all done briefly before reset". The visible symptom was {progress:{percent:100, complete:true}} immediately after POST /search (queue-empty window) followed by progress=0 a few seconds later (mid-run), then nothing useful at completion (raw resets to 0). The lifecycle now lives in the refresher: - POST /search calls m_state.MarkSearchStarted(kind) — wipes the results map, sets active=true with the requested kind. - RefresherTick polls EC_OP_SEARCH_RESULTS + _PROGRESS every tick while active=true and feeds the raw value, results-count, and now into AdvanceSearchProgress. - AdvanceSearchProgress is a pure function in Refresher.cpp that state-machines the raw value into normalized (percent, complete, active, saw_in_progress). Disambiguates via the kind, prior saw_in_progress, results-map size, and a 30 s defensive timeout. Kad bumps started_at on each new result so a productive Kad search isn't cut off; saw_in_progress excludes the timeout for a genuinely climbing global. - GET /search/results reads straight from state (no more TtlCache, no on-demand EC fetches). Surface now includes progress.active. EventDiff publishes search_result_added per new ECID in the results map and search_finished on the complete=false->true transition. The baseline is gated on search_initialised so a fresh tick after MarkSearchStarted doesn't double-fire. RefresherTest grows 12 cases covering every state-machine branch: global climb-then-reset, raw=100 caught vs missed, queue-empty mask, fast-complete (missed climb), local 0xffff path, local raw=0 finalize, Kad raw=0 stays active, Kad 0xfffe completion, Kad deadline extension on new result, timeout after 30 s of silence, timeout skipped for climbing global, inactive-state graceful exit. 19-search regression assertion: progress.complete must be false immediately after POST /search. 22-sse-diff-emission gains a search_result_added / search_finished smoke using a local search (works even on a fully-disconnected daemon). Reported in #201 review. * docs(amuleapi): declare the search SSE channel + tighten filter tests The prefix-based event_channel fallback in Api.cpp already routes search_result_added / search_finished to channel "search", so ?channels=search works in practice. Make it explicit so the mapping table stays a single source of truth alongside docs/api/EVENTS.md and a future event family doesn't end up routed via the silent fallback. EVENTS.md: add the search channel row to the channels table and catalog both event payloads with the per-search ECID-stability note (amuled assigns a fresh ECID via CECID's monotonic s_IDCounter on every new search, even for the same file across repeated queries). 26-rfc-followup-endpoints.sh: the channels=downloads,status exclusion check now also forbids search_*; a new positive check asserts channels=search delivers search_finished and excludes download / status / etc. Refs #201 review. * refactor(amuleapi): unify downloads + shared into one ECID-keyed FileSnapshot amuled's downloads (CPartFile in downloadqueue) and shared (CKnownFile in sharedfiles + partfiles with PARTFILE_SHARED=true) views share a single per-process ECID counter (CECID::s_IDCounter) and a unified encoder-side CObjTagMap. Mirroring that on the receiver side avoids the class of bugs we kept patching: - The /shared "ghost" rows we fixed in e57cbf9 were caused by the shared walker hitting an ECID that the downloads walker had already populated in a SEPARATE m_shared cache, with the PARTFILE_HASH child tag CValueMap-suppressed (already sent on a prior partfile-walker tick). We worked around it by composing a dl_identity_fallback map and passing it to the shared walker. - With one ECID-keyed FileSnapshot map, the shared walker finds the entry the downloads walker just touched, with hash + name already set. It just flips is_shared=true and merges its own fields. The fallback compose disappears. Schema: struct FileSnapshot { uint32_t ecid; string hash, name, ed2k_link; uint64_t size; string priority; bool is_downloading, is_shared; struct DownloadSide { ... } download; // partfile-walker fields struct SharedSide { ... } shared; // knownfile-walker fields }; Role transitions: - Downloads walker: on PARTFILE entry, sets is_downloading=true and merges f.download. On FILE_REMOVED, clears is_downloading + resets download to default; drops the entry entirely if !is_shared. - Shared walker: same shape for is_shared/f.shared. PARTFILE_SHARED= false clears is_shared and resets the shared sub-block; the entry stays in the unified map (entity eviction is FILE_REMOVED's job). - Order-independent: running downloads-then-shared and shared-then- downloads converge to the same final state. Reset-on-transition guarantees /downloads can never serve stale upload stats, and /shared can never serve stale download stats — same lookup hitting either view always reads from a freshly-walked sub-block. API surface unchanged: GET /downloads filters m_files by is_downloading; GET /shared filters by is_shared. Same wire shape; clients see no difference. Bonus from the unified map: FindDownload(hash) / FindShared(hash) now hit a parallel std::unordered_map index (m_hash_to_ecid) that's rebuilt at the end of every MutateDownloads / MutateShared. Was O(N) linear scan; now O(1) hash lookup + O(log N) map find + role-flag check. REFERENCE.md updated. RefresherTest migrated to FileSnapshot + role-flag assertions (32 cases, all pass). EventDiff diffs the unified files map with role-flag awareness: download_{added,updated,removed} fires on is_downloading transitions, shared_{added,updated,removed} on is_shared transitions. A single tick can emit both for one ECID (e.g. partfile becoming shared). Full curl suite green: 27/27. Refs #201 review. * refactor(amuleapi): drop file ECIDs from the public API surface Hashes (MD4, 32-char hex) are the only identifier the API exposes for partfiles and shared files. File ECIDs were never wire-public: - /downloads/{key} → /downloads/{hash} (PATCH/DELETE/detail) - /shared/{key} → /shared/{hash} - removed "ecid" from /downloads, /shared, search-result event payloads - /clients: upload_file_ecid / download_file_ecid → upload_file_hash / download_file_hash. The walker now resolves EC_TAG_CLIENT_UPLOAD_FILE and EC_TAG_CLIENT_REQUEST_FILE into MD4 hashes at refresher-tick time using an ECID→hash snapshot built from the unified file map. Client and server ECIDs stay — they are still the URL key for /clients/{ecid} and /servers/{ecid}, and there's no hash for either. Tests updated: curl phases 04 + 17 drop the file-ECID assertions and the PATCH /shared/{ECID} round-trip. All 22 cpp unit tests still pass; all 27 curl phases still pass against the local amuled. Closes the file-ECID strip discussed in #201. * refactor(amuleapi): rename clients.ecid → client_ecid Disambiguates from the file ECIDs we just stripped — `ecid` alone made it look like a file identifier, when it's actually the peer's identity. Affected surfaces: - GET /clients[] payload - client_added / client_updated SSE payloads - client_removed SSE payload (`{ "client_ecid": N }`) server_ecid stays as plain `ecid` — same field name as before, still the URL key for /servers/{ecid}. Curl phases 10 + 26 updated. All 22 cpp unit tests + 4 affected curl phases (10, 22, 24, 26) pass. * perf(amuleapi): switch m_files to unordered_map with inline hash index CState's unified file map was std::map with a parallel std::unordered_map hash→ECID index that got rebuilt from scratch on every MutateDownloads / MutateShared call (one rebuild per refresher tick). That O(N) per-tick scan was the dominant cost on large libraries — 100k files × 2 walkers × 1 Hz = 200k iterations per second spent re-deriving an index that almost never changed. Replace both with a single FileMap wrapper class that owns both data structures: * std::unordered_map for ECID-keyed storage * std::unordered_map for hash → ECID lookups emplace() and erase() update the index inline (O(1) avg per change); hash never mutates within an ECID so no other surface needs hooks. Walkers keep their .find()/.emplace()/.erase() calls unchanged — the wrapper exposes the same iterator surface. Per-tick rebuild deleted. Lookups now both O(1) avg: * ECID → FileSnapshot : FileMap::find(ecid) * hash → FileSnapshot : FindEcidByHash(hash) + find(ecid) CState::Downloads() / Shared() / Files() vectors are no longer in ECID-ascending order (unordered_map iteration is bucket-defined). Consumers already sort on their side (ngosang's feedback on PR #201), so this is wire-visible but harmless. REFERENCE.md tidied — the previous /downloads/{key} prose was left over from the file-ECID-strip pass; consistent {hash} naming throughout now, no internal-state details leaking onto the public API surface. 22/22 cpp unit tests + 27/27 curl phases pass. * docs(amuleapi): add SSE bootstrap section (open stream then snapshot) EVENTS.md never spelled out the call ordering for the snapshot + stream combination. The naïve REST-first → SSE-second pattern silently drops events that fire between the two — a one-line bootstrap bug that's easy to write and hard to spot in testing. New "Bootstrap: snapshot + stream" section explicitly: - Names the race (REST→SSE loses events fired in the gap) - Documents the correct order (open SSE first, then GET, apply events as upserts on top of the snapshot) - Explains why the merge is consistent (events for tick N are emitted after the state mutation for tick N is committed) - Gives a JS bootstrap example with parallel snapshot fetches - Points at the resync recovery rules for the failure modes * docs(amuleapi): fill SSE doc gaps + convert § cross-refs to anchor links Bootstrap-companion gaps the doc didn't cover, found while sanity-checking the bootstrap example against real-world client paths: - Connecting / Browser example: add an `error` listener with the `readyState === CLOSED` terminal-failure check, and clarify that the native browser EventSource API can't carry an `Authorization` header so browser bearer-on-SSE requires a polyfill. - Channels and filtering: note that the snapshot fetches in the bootstrap must mirror the `?channels=` filter; pulling unsubscribed collections leaves them silently stale. - Event catalog intro: call out that `status_changed` is a full envelope replacement and `log_appended` is an append — they don't fit the `_added` / `_updated` / `_removed` upsert pattern. - Search / Logs channel sections: explicit pointer back to Bootstrap noting that `/search/results` and `/logs/amule` aren't fetched there by default, with the case for adding them. Plus a tidy-up: every `§Section` cross-ref now uses a clickable markdown anchor link instead of the section-symbol convention (matches the REFERENCE.md links and renders properly on GitHub). * feat(ec): unambiguous search lifecycle tags on EC_OP_SEARCH_PROGRESS The existing EC_TAG_SEARCH_STATUS overloads a single uint32 to mean a lot of different things — raw=0 covers "no search", "Kad in progress", "global queue not yet populated", and "global just finished"; raw=100 covers "global queue empty at start" and "global all done briefly". 0xfffe = Kad done, 0xffff = global done (or local in progress). The desktop GUI and amulecmd live with this because their UI state + explicit notifies cover the gaps; for headless consumers driving a JSON wire surface (amuleapi) it forces a defensive state machine (saw_in_progress tracking, 30 s timeout, started_at bump-on-result). Add three new optional tags on the EC_OP_SEARCH_PROGRESS response so modern consumers can read the lifecycle directly without the sentinel decode, while pre-3.1 consumers (amulegui, amulecmd, amuleweb) keep reading EC_TAG_SEARCH_STATUS unchanged: EC_TAG_SEARCH_LIFECYCLE_STATE uint8 0=idle / 1=running / 2=finished EC_TAG_SEARCH_LIFECYCLE_KIND uint8 0=local / 1=global / 2=kad EC_TAG_SEARCH_RESULT_COUNT uint32 size of m_results[m_currentSearch] Derivation in CSearchList::GetSearchLifecycleState(): - m_currentSearch == -1 → IDLE (no search this session) - m_searchType == KadSearch → m_KadSearchFinished ? FINISHED : RUNNING - otherwise (Local / Global) → m_searchInProgress ? RUNNING : FINISHED EC_TAG_SEARCH_STATUS is unchanged. amulegui / amulecmd / amuleweb iterate tags they recognise and skip unknowns — verified by a full build of all four binaries (amule, amuled, amulegui, amulecmd) against the same EC core. No per-searchID tracking yet — Get_EC_Response_Search still pins the sentinel 0xffffffff. Lifting that is a separate (larger) change. * refactor(amuleapi): consume EC_TAG_SEARCH_LIFECYCLE_STATE directly amuleapi pins a daemon version carrying the new EC_TAG_SEARCH_LIFECYCLE_* tags landed in the previous commit, so there's no reason to keep the sentinel-disambiguation state machine around as a fallback. Replace it with a single ten-line mapping from (lifecycle_state, raw_pct) to (active, complete, percent). Refresher.cpp/h: AdvanceSearchProgressFromLifecycle becomes the only AdvanceSearchProgress. The legacy state machine, its 60-line comment block, and the defensive 30 s timeout / saw_in_progress / Kad-deadline- bump-on-result tricks are all gone. RefresherTick.cpp: drops the two-path branch that picked between legacy and lifecycle. Reads both EC_TAG_SEARCH_STATUS (for the global percent) and EC_TAG_SEARCH_LIFECYCLE_STATE every tick; passes both to the helper. State.h: SearchProgressSnapshot loses saw_in_progress, raw, started_at, last_results_count — all of which only existed to drive the old disambiguation. Keeps active / kind / percent / complete (the four fields surfaced by GET /search/results and the search_finished S…

Uh oh!

Clarify relation between this repository and the original one #1

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions