Skip to content

KnownFileList: close TOCTOU UAF window in Save() / PruneDuplicates (#685) - #690

Merged
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/known-file-list-prune-uaf
May 23, 2026
Merged

KnownFileList: close TOCTOU UAF window in Save() / PruneDuplicates (#685)#690
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/known-file-list-prune-uaf

Conversation

@got3nks

@got3nks got3nks commented May 23, 2026

Copy link
Copy Markdown
Contributor

Fixes #685.

Root cause

CKnownFileList::Save() snapshotted the sharedfiles "in-use" set without holding its own lock, deferring the lock acquisition until just before PruneDuplicates. Between snapshot and prune, the main thread (or any other thread that calls CSharedFileList::AddFile) could enter a CKnownFile into sharedfiles that the snapshot missed. PruneDuplicates then ran isProtected() against the stale set, saw the file unprotected, and deleted it — while sharedfiles->m_Files_map[hash] still pointed at it.

Sharedfiles then handed the dangling pointer back to CFileEncoderMap::UpdateEncoders(), and Get_EC_Response_GetUpdate built CEC_PartFile_Tag on the freed CKnownFile. By the time the freed slot had been reclaimed for some unrelated string allocation, the tag ctor accessed file->m_pAICHHashSet, found garbage bytes, and segfaulted.

@JamesOlvertone's gdb dump on #685 confirmed this concretely — the freed slot held header-path strings (/usr/include/cryptopp/..., c++/15/bits/... from a wxString / __FILE__ buffer):

_vptr.CAbstractFile = 0x636e692f7273752f  -> "/usr/inc"
m_abyFileHash        = "lude/cryptopp/se"
m_strComment.m_str   = 0x7972632f6564756c  -> "lude/cry"
m_pAICHHashSet       = 0x737469622f35312f  -> "/15/bits"

A separate-but-adjacent race exists for downloadqueue: a CPartFile being uploaded can be removed from sharedfiles by CUploadDiskIOThread::Entry on CFile::Open failure, but remains in downloadqueue. Pruning by sharedfiles snapshot alone misses it.

Fix

Two changes in Save() / PruneDuplicates:

  1. Take list_mut FIRST, then snapshot sharedfiles AND downloadqueue under it. This makes the "in-use" set authoritative at the moment PruneDuplicates starts. The brief overlap of knownfiles → sharedfiles and knownfiles → downloadqueue locks is safe: no code path in the project takes them in the reverse order while holding the first. (Sharedfiles never calls into knownfiles under its own lock; downloadqueue never calls into knownfiles at all. The pre-existing comment in Save() warning about ABBA with SafeAddKFile was over-cautious — SafeAddKFile holds sharedfiles' list_mut only inside AddFile, never nested with knownfiles' list_mut.)

  2. Re-validate every live-entry candidate in PruneDuplicates Pass 3 immediately before deletion via sharedfiles->GetFileByID(hash) and downloadqueue->GetFileByID(hash). Even after change 1, the sharedfiles/downloadqueue locks are released between snapshot and the prune body, so a concurrent SafeAddKFile or RemoveFile in that interval could still race. The per-candidate re-query under the owner's lock makes the protection point-in-time correct.

The Pass 3 re-query cost is one map.find per dead candidate (a handful per Save in practice) — negligible vs. the I/O cost Save was already paying.

Test

Local macOS build of amule amuled amulegui amuleweb — clean. The race itself is hard to trigger synthetically (timing-dependent), but the static lock-order argument holds: with change 1 in place, the only race left is the gap between snapshot release and Pass 3 entry, which change 2 closes with the per-delete re-check.

CKnownFileList::Save() snapshotted the sharedfiles "in-use" set
WITHOUT holding its own lock, deferring the lock acquisition until
right before PruneDuplicates. Between snapshot and prune, the main
thread (or any other thread that calls CSharedFileList::AddFile)
could enter a CKnownFile into sharedfiles that the snapshot had
missed. PruneDuplicates then ran isProtected() against the stale
inUse set, saw the file unprotected, and "delete"d it -- while
sharedfiles->m_Files_map[hash] still pointed at it.

Sharedfiles then continued handing the dangling pointer back to any
caller, most prominently CFileEncoderMap::UpdateEncoders() ->
Get_EC_Response_GetUpdate() loop, which built CEC_PartFile_Tag
on the freed CKnownFile. By the time the freed slot had been
reclaimed for some unrelated string allocation, the tag ctor
accessed `file->m_pAICHHashSet`, found garbage bytes (in the
reported amule-project#685 dump, the slot held header-path strings from a
wxString / __FILE__ buffer), and segfaulted.

A separate but adjacent race exists with downloadqueue: a
CPartFile being uploaded can be removed from sharedfiles by
CUploadDiskIOThread::Entry on CFile::Open failure, but it remains
in the downloadqueue. Pruning by sharedfiles snapshot alone would
miss it.

Two changes:

1. Take list_mut FIRST, then snapshot both sharedfiles and
   downloadqueue under it. This makes the "in-use" set authoritative
   at the moment PruneDuplicates starts. The brief overlap of
   knownfiles -> sharedfiles and knownfiles -> downloadqueue locks
   is safe: no code path in the project takes them in the reverse
   order while holding the first. (sharedfiles never calls into
   knownfiles under its own lock; downloadqueue never calls into
   knownfiles at all.) The previous comment warning about ABBA with
   CSharedFileList::SafeAddKFile turns out to be wrong on inspection
   of the current code -- SafeAddKFile holds sharedfiles' list_mut
   only inside AddFile, never nested with knownfiles' list_mut.

2. Re-validate every live-entry candidate in PruneDuplicates Pass 3
   immediately before deletion via sharedfiles->GetFileByID(hash)
   and downloadqueue->GetFileByID(hash). Even after step 1, the
   sharedfiles / downloadqueue locks are released between snapshot
   and the prune body, so a concurrent SafeAddKFile or RemoveFile
   in that interval could still race. The per-candidate re-query
   under the owner's lock makes the protection point-in-time
   correct.

The Pass 3 re-query cost is one map lookup per dead candidate (a
handful per Save call in practice) and the lookup hits map.find,
not the full map iteration -- negligible vs. the I/O cost Save was
already paying.

Fixes amule-project#685.
@mrjimenez
mrjimenez merged commit 094ce2a into amule-project:master May 23, 2026
12 checks passed
@got3nks
got3nks deleted the fix/known-file-list-prune-uaf branch May 23, 2026 20:06
ngosang added a commit to ngosang/amule that referenced this pull request Jul 29, 2026
… columns (amule-project#690)

Surface `country_code` (amule-project#439/amule-project#440, added core-side in 5b7cdf1) in the Web UI as
a plain 2-letter ISO 3166-1 alpha-2 code, no flag: a sortable country column in
the peer table (Clients page + the per-file Clients tab of both detail panels)
and in the ED2K server table. Its header is an abbreviation ("CC" in English,
"CP" in Spanish) because a spelled-out "Country" would force ~80px for a 2-char
cell.

The column always renders. The daemon omits the EC tag entirely when GeoIP is off
or the build lacks it, which reaches the REST layer as `country_code: ""` and
shows as "—" like any other empty cell -- so nothing has to probe for GeoIP
support (/status carries no GeoIP flag, and the Web UI has no preferences store
to read `ip2country.supported` from).

Also fills three gaps in the peer table -- fields /clients has always returned
but nothing painted: Address (`ip:port`), OS (`os_info`) and User hash. All three
start hidden in every consumer, so they only widen the table once picked from the
column picker. Peer columns now run CC, Address, Name, User hash, Software, OS,
File, so each identity field sits next to the one it qualifies. The four transfer
totals are relabelled to the compact DL/UL total and DL/UL session, which also
lets all four share one width.

The server table is reordered to match (CC, Address, Name, Description, Users,
Files, Version, Ping, Priority, Actions) and starts with Address, Version and
Ping hidden, keeping the default view to what identifies a server and how busy it
is. Description also drops its fixed 180px and splits the leftover width with
Name, since descriptions are long (forum URLs, blurbs). Both tables' Name columns
lose their `always` flag, so the picker offers them like any other column -- with
Address and CC available, the name is no longer the only way to tell rows apart.
Downloads / Shared / Search keep theirs pinned: there the name column anchors the
row-selection checkbox.

Two pre-existing bugs this surfaced, both fixed for every list view:

  * A header label wider than its column spilled over the neighbouring header
    instead of being clipped (`white-space: nowrap`, and no overflow rule of its
    own) -- which is why a "Country" label looked like it fit 70px. Headers now
    ellipsize and carry the full label as a title tooltip, so a translation that
    outgrows its column stays readable on hover.
  * The server Address column sorted its `ip:port` strings lexically, putting
    10.x before 9.x and .182 before .87. Both tables now sort it by IP value
    through one shared `ipNum` helper in table.js, next to the other sort/filter
    helpers.

Peer detail-only fields (GET /clients/{ecid}) are untouched: they need a peer
detail panel, not a column.
mrjimenez pushed a commit to mrjimenez/amule that referenced this pull request Jul 30, 2026
…mule-project#694)

`country_code` reaches the Web UI on /clients, /servers and their SSE
diffs, but the flag image had nowhere to come from, so amule-project#690 could only
paint the bare 2-letter code. This is the delivery mechanism for the
artwork, and it adds no second copy of it: the famfamfam flags are
already compiled into a byte table by src/icons/embed_icons.py for the
desktop GUI, so amuleapi links the same generated `icon_data.c` and
serves the flag_<cc> entries straight out of .rodata.

Nothing touches the file system, so the route behaves identically in a
source-tree dev run, an installed layout, and an API-only deployment
with StaticRoot unset — and it has no path-traversal surface to guard
beyond the code itself. `{code}` is two lowercase ASCII letters, or the
literal `unknown` for the "??" placeholder CCountryFlags falls back to,
so a frontend can match what the desktop list draws for an unresolved
country. Anything else is a 404, including a well-formed code the set
has no artwork for. The art id is built by concatenating "flag_" with
the code, so that whitelist — not the upstream traversal gate — is what
keeps a crafted code from naming a non-flag icon in the shared table.

ETag, If-None-Match -> 304 and HEAD body-stripping come for free from
the existing response post-processing; the handler only adds a one-day
Cache-Control so a peer list full of <img> tags doesn't issue one
conditional request per country on every reload.

Responses whose media type is already entropy-coded (PNG, JPEG, GIF,
WebP, woff/woff2, zip, gzip) now skip the gzip encoder. Deflate over a
PNG buys nothing and routinely grows the body; before this the 700-900
byte flags cleared the 256-byte compression threshold and got encoded
for no gain.

Cost is ~320 KB of read-only data in the amuleapi binary — the whole
icon table, of which only the flags are ever asked for. Trimming it to
flags only would need a second embed_icons.py invocation and its own
generated TU; the unused entries are pages that never fault in, which
is not worth a parallel asset pipeline.

Verified on macOS: all 249 files under src/icons/flags/ come back
byte-identical over the route, and the new 32-country-flags regtest
(29 assertions) covers content type, PNG magic, ETag/304, HEAD, the
no-gzip guarantee, the "unknown" placeholder, malformed and traversal
codes, 405 on non-safe methods, and that no auth is required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

crash after renaming complete file

2 participants