KnownFileList: close TOCTOU UAF window in Save() / PruneDuplicates (#685) - #690
Merged
mrjimenez merged 1 commit intoMay 23, 2026
Merged
Conversation
CKnownFileList::Save() snapshotted the sharedfiles "in-use" set WITHOUT holding its own lock, deferring the lock acquisition until right before PruneDuplicates. Between snapshot and prune, the main thread (or any other thread that calls CSharedFileList::AddFile) could enter a CKnownFile into sharedfiles that the snapshot had missed. PruneDuplicates then ran isProtected() against the stale inUse set, saw the file unprotected, and "delete"d it -- while sharedfiles->m_Files_map[hash] still pointed at it. Sharedfiles then continued handing the dangling pointer back to any caller, most prominently CFileEncoderMap::UpdateEncoders() -> Get_EC_Response_GetUpdate() loop, which built CEC_PartFile_Tag on the freed CKnownFile. By the time the freed slot had been reclaimed for some unrelated string allocation, the tag ctor accessed `file->m_pAICHHashSet`, found garbage bytes (in the reported amule-project#685 dump, the slot held header-path strings from a wxString / __FILE__ buffer), and segfaulted. A separate but adjacent race exists with downloadqueue: a CPartFile being uploaded can be removed from sharedfiles by CUploadDiskIOThread::Entry on CFile::Open failure, but it remains in the downloadqueue. Pruning by sharedfiles snapshot alone would miss it. Two changes: 1. Take list_mut FIRST, then snapshot both sharedfiles and downloadqueue under it. This makes the "in-use" set authoritative at the moment PruneDuplicates starts. The brief overlap of knownfiles -> sharedfiles and knownfiles -> downloadqueue locks is safe: no code path in the project takes them in the reverse order while holding the first. (sharedfiles never calls into knownfiles under its own lock; downloadqueue never calls into knownfiles at all.) The previous comment warning about ABBA with CSharedFileList::SafeAddKFile turns out to be wrong on inspection of the current code -- SafeAddKFile holds sharedfiles' list_mut only inside AddFile, never nested with knownfiles' list_mut. 2. Re-validate every live-entry candidate in PruneDuplicates Pass 3 immediately before deletion via sharedfiles->GetFileByID(hash) and downloadqueue->GetFileByID(hash). Even after step 1, the sharedfiles / downloadqueue locks are released between snapshot and the prune body, so a concurrent SafeAddKFile or RemoveFile in that interval could still race. The per-candidate re-query under the owner's lock makes the protection point-in-time correct. The Pass 3 re-query cost is one map lookup per dead candidate (a handful per Save call in practice) and the lookup hits map.find, not the full map iteration -- negligible vs. the I/O cost Save was already paying. Fixes amule-project#685.
This was referenced May 23, 2026
Closed
ngosang
added a commit
to ngosang/amule
that referenced
this pull request
Jul 29, 2026
… columns (amule-project#690) Surface `country_code` (amule-project#439/amule-project#440, added core-side in 5b7cdf1) in the Web UI as a plain 2-letter ISO 3166-1 alpha-2 code, no flag: a sortable country column in the peer table (Clients page + the per-file Clients tab of both detail panels) and in the ED2K server table. Its header is an abbreviation ("CC" in English, "CP" in Spanish) because a spelled-out "Country" would force ~80px for a 2-char cell. The column always renders. The daemon omits the EC tag entirely when GeoIP is off or the build lacks it, which reaches the REST layer as `country_code: ""` and shows as "—" like any other empty cell -- so nothing has to probe for GeoIP support (/status carries no GeoIP flag, and the Web UI has no preferences store to read `ip2country.supported` from). Also fills three gaps in the peer table -- fields /clients has always returned but nothing painted: Address (`ip:port`), OS (`os_info`) and User hash. All three start hidden in every consumer, so they only widen the table once picked from the column picker. Peer columns now run CC, Address, Name, User hash, Software, OS, File, so each identity field sits next to the one it qualifies. The four transfer totals are relabelled to the compact DL/UL total and DL/UL session, which also lets all four share one width. The server table is reordered to match (CC, Address, Name, Description, Users, Files, Version, Ping, Priority, Actions) and starts with Address, Version and Ping hidden, keeping the default view to what identifies a server and how busy it is. Description also drops its fixed 180px and splits the leftover width with Name, since descriptions are long (forum URLs, blurbs). Both tables' Name columns lose their `always` flag, so the picker offers them like any other column -- with Address and CC available, the name is no longer the only way to tell rows apart. Downloads / Shared / Search keep theirs pinned: there the name column anchors the row-selection checkbox. Two pre-existing bugs this surfaced, both fixed for every list view: * A header label wider than its column spilled over the neighbouring header instead of being clipped (`white-space: nowrap`, and no overflow rule of its own) -- which is why a "Country" label looked like it fit 70px. Headers now ellipsize and carry the full label as a title tooltip, so a translation that outgrows its column stays readable on hover. * The server Address column sorted its `ip:port` strings lexically, putting 10.x before 9.x and .182 before .87. Both tables now sort it by IP value through one shared `ipNum` helper in table.js, next to the other sort/filter helpers. Peer detail-only fields (GET /clients/{ecid}) are untouched: they need a peer detail panel, not a column.
mrjimenez
pushed a commit
to mrjimenez/amule
that referenced
this pull request
Jul 30, 2026
…mule-project#694) `country_code` reaches the Web UI on /clients, /servers and their SSE diffs, but the flag image had nowhere to come from, so amule-project#690 could only paint the bare 2-letter code. This is the delivery mechanism for the artwork, and it adds no second copy of it: the famfamfam flags are already compiled into a byte table by src/icons/embed_icons.py for the desktop GUI, so amuleapi links the same generated `icon_data.c` and serves the flag_<cc> entries straight out of .rodata. Nothing touches the file system, so the route behaves identically in a source-tree dev run, an installed layout, and an API-only deployment with StaticRoot unset — and it has no path-traversal surface to guard beyond the code itself. `{code}` is two lowercase ASCII letters, or the literal `unknown` for the "??" placeholder CCountryFlags falls back to, so a frontend can match what the desktop list draws for an unresolved country. Anything else is a 404, including a well-formed code the set has no artwork for. The art id is built by concatenating "flag_" with the code, so that whitelist — not the upstream traversal gate — is what keeps a crafted code from naming a non-flag icon in the shared table. ETag, If-None-Match -> 304 and HEAD body-stripping come for free from the existing response post-processing; the handler only adds a one-day Cache-Control so a peer list full of <img> tags doesn't issue one conditional request per country on every reload. Responses whose media type is already entropy-coded (PNG, JPEG, GIF, WebP, woff/woff2, zip, gzip) now skip the gzip encoder. Deflate over a PNG buys nothing and routinely grows the body; before this the 700-900 byte flags cleared the 256-byte compression threshold and got encoded for no gain. Cost is ~320 KB of read-only data in the amuleapi binary — the whole icon table, of which only the flags are ever asked for. Trimming it to flags only would need a second embed_icons.py invocation and its own generated TU; the unused entries are pages that never fault in, which is not worth a parallel asset pipeline. Verified on macOS: all 249 files under src/icons/flags/ come back byte-identical over the route, and the new 32-country-flags regtest (29 assertions) covers content type, PNG magic, ETag/304, HEAD, the no-gzip guarantee, the "unknown" placeholder, malformed and traversal codes, 405 on non-safe methods, and that no auth is required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #685.
Root cause
CKnownFileList::Save()snapshotted the sharedfiles "in-use" set without holding its own lock, deferring the lock acquisition until just beforePruneDuplicates. Between snapshot and prune, the main thread (or any other thread that callsCSharedFileList::AddFile) could enter aCKnownFileinto sharedfiles that the snapshot missed.PruneDuplicatesthen ranisProtected()against the stale set, saw the file unprotected, anddeleted it — whilesharedfiles->m_Files_map[hash]still pointed at it.Sharedfiles then handed the dangling pointer back to
CFileEncoderMap::UpdateEncoders(), andGet_EC_Response_GetUpdatebuiltCEC_PartFile_Tagon the freedCKnownFile. By the time the freed slot had been reclaimed for some unrelated string allocation, the tag ctor accessedfile->m_pAICHHashSet, found garbage bytes, and segfaulted.@JamesOlvertone's gdb dump on #685 confirmed this concretely — the freed slot held header-path strings (
/usr/include/cryptopp/...,c++/15/bits/...from awxString/__FILE__buffer):A separate-but-adjacent race exists for downloadqueue: a
CPartFilebeing uploaded can be removed from sharedfiles byCUploadDiskIOThread::EntryonCFile::Openfailure, but remains in downloadqueue. Pruning by sharedfiles snapshot alone misses it.Fix
Two changes in
Save()/PruneDuplicates:Take
list_mutFIRST, then snapshot sharedfiles AND downloadqueue under it. This makes the "in-use" set authoritative at the momentPruneDuplicatesstarts. The brief overlap ofknownfiles → sharedfilesandknownfiles → downloadqueuelocks is safe: no code path in the project takes them in the reverse order while holding the first. (Sharedfiles never calls into knownfiles under its own lock; downloadqueue never calls into knownfiles at all. The pre-existing comment inSave()warning about ABBA withSafeAddKFilewas over-cautious —SafeAddKFileholds sharedfiles'list_mutonly insideAddFile, never nested with knownfiles'list_mut.)Re-validate every live-entry candidate in
PruneDuplicatesPass 3 immediately before deletion viasharedfiles->GetFileByID(hash)anddownloadqueue->GetFileByID(hash). Even after change 1, the sharedfiles/downloadqueue locks are released between snapshot and the prune body, so a concurrentSafeAddKFileorRemoveFilein that interval could still race. The per-candidate re-query under the owner's lock makes the protection point-in-time correct.The Pass 3 re-query cost is one
map.findper dead candidate (a handful perSavein practice) — negligible vs. the I/O costSavewas already paying.Test
Local macOS build of
amule amuled amulegui amuleweb— clean. The race itself is hard to trigger synthetically (timing-dependent), but the static lock-order argument holds: with change 1 in place, the only race left is the gap between snapshot release and Pass 3 entry, which change 2 closes with the per-delete re-check.