sharedfiles: auto-rescan via wxFileSystemWatcher - #591
Merged
mrjimenez merged 2 commits intoMay 13, 2026
Conversation
aMule today only re-scans the on-disk content of shareddir_list on
explicit user action ("Reload shared files") or at daemon start.
Dropping a file into a shared folder leaves it un-shared until the
next manual reload — a long-standing user-experience gap for anyone
who curates a library outside aMule (rsync, Finder drag-and-drop,
torrent client move-on-complete, etc.).
Add a directory watcher that observes every path in shareddir_list
via wxFileSystemWatcher (inotify on Linux, FSEvents on macOS,
ReadDirectoryChangesW on Windows, kqueue on BSD) and triggers a
debounced CSharedFileList::Reload when something changes. The
debounce window is 5 s, long enough to coalesce the bursts that
bulk operations (tar -x, rsync) produce into one Reload, short
enough that a single drag-and-drop feels responsive.
Auto-share new subdirectories of any watched parent. When a user
creates Music/2026/Album under an already-shared Music tree, the
new subdir is appended to shareddir_list and registered with the
watcher, mirroring what users expect from the existing recursive-
share flow. The change is persisted via a new
CPreferences::SaveSharedFolders that writes only shareddir.dat so
we don't force a full preferences.dat rewrite on every mkdir event.
New CSharedDirWatcher class (src/SharedDirWatcher.{h,cpp}) owns the
wxFileSystemWatcher and the debounce wxTimer. CSharedFileList owns
the watcher lifetime via a new EnableDirectoryWatcher(bool) entry
point, called once at daemon start (gated on the new pref) and again
when the user toggles the pref live from PrefsUnifiedDlg. Reload
calls watcher->Refresh after rebuilding the list, so dirs added or
removed since the previous Reload are picked up without having to
diff the path set ourselves.
Deferred watcher construction. wx's inotify backend (Linux) requires
an active wx event loop at wxFileSystemWatcher::Init() time —
without one, m_service stays null and every Add() silently returns
false. CamuleApp::OnInit() runs before the loop starts, so the
first Enable() call from there queues itself via wxEvtHandler's
CallAfter and re-runs at the next loop iteration. Calls from the
prefs dialog or EC apply path already run inside an active loop and
take the immediate branch. Queueing on the CSharedDirWatcher itself
(rather than wxTheApp) lets wx purge the pending event if the
watcher is destroyed before the loop drains.
Two registration strategies. Linux (inotify), Windows
(ReadDirectoryChangesW), and BSD (kqueue) use per-directory Add() —
shareddir_list already enumerates every subdirectory individually
when a user uses the recursive-share button, so inotify watch
count tracks shareddir_list.size() rather than total subtree
depth. macOS routes through AddTree() instead, because (a) wx
3.3.2's bare Add() falls through to the kqueue base and returns
false on otherwise-openable directories — verified against wx
upstream and reproduced locally — and (b) FSEvents is the API
Apple actually recommends for directory monitoring. Since
AddTree() is recursive on macOS, the registration loop skips
shareddir_list entries whose ancestor is also in the list so we
don't try to open overlapping FSEvents streams (wx silently
returns false on the second). The corresponding RegisterNew-
Subdirectory path is a no-op on the watcher under macOS — the
parent stream already delivers events for new descendants —
while other platforms still need an explicit Add() for the new
subdir's own contents.
The feature is exposed as a new pref AutoRescanSharedDirs, defaulted
ON so the behaviour is visible without opt-in. A checkbox is added
to the Directories preferences panel
(IDC_AUTO_RESCAN_SHARED = 10333, picked above the wxDesigner-
generated ID range so a future muuli_wdr.* regeneration doesn't
collide). Users on Linux hosts hitting /proc/sys/fs/inotify/
max_user_watches can disable it; the watcher already logs and
continues on per-path Add() failure so partial coverage is the
graceful-degradation path rather than a hard error.
A new EC tag EC_TAG_DIRECTORIES_AUTO_RESCAN (0x1A05) carries the
pref over the EC channel so amulegui can flip it on a remote
amuled. The amuled-side Apply path calls EnableDirectoryWatcher
immediately so the change takes effect without a daemon restart.
amule-remote-gui provides a no-op EnableDirectoryWatcher on the
GUI side — the watcher only ever lives on the file-owning daemon.
Build: amule + amuled + amulecmd + amulegui compile and link clean
on macOS Apple Silicon (Ninja, monolithic + daemon + remotegui +
amulecmd targets). End-to-end verified on macOS: dropping a file
into a watched directory triggers FSEvents → 5 s debounce →
Reload → file appears in the shared-files view.
amuled on macOS registers wxFsEventsFileSystemWatcher streams just fine, but never receives any of their callbacks: the stream is scheduled on the calling thread's CFRunLoop, and wxAppConsole's event loop on macOS does not spin that runloop. Under aMule.app a real Cocoa main loop drives it, so the GUI path has always worked; under amuled the events queue forever and the auto-rescan feature is silently inert. Add a 200 ms wxTimer in CSharedDirWatcher that calls CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0, true) on each tick. That is a non-blocking drain: it dispatches any FSEvents callbacks the kernel has queued on this thread's CFRunLoop and returns immediately if there are none. wx's wrapper then translates those callbacks into wxFileSystemWatcherEvent and posts them through the normal event queue, where our existing OnFileSystemEvent handler picks them up and runs the 5 s debounce as on every other platform. The pump only starts when wxTheApp->IsGUI() is false, so aMule.app (whose Cocoa main loop already pumps the runloop) is unaffected. Cost in the amuled case is ~5 non-blocking wakeups per second on the main thread; CFRunLoopRunInMode with a zero timeout returns immediately when no sources are ready, so the steady-state overhead is negligible relative to amuled's existing per-second timers. The pump is started in Enable() after the watcher is up and stopped in Disable() before the watcher is torn down, so toggling the pref from amulegui via EC reflects on the runloop drain as well. Build: amule + amuled compile clean on macOS Apple Silicon. End-to-end verified on macOS amuled: dropping a file into a watched directory triggers FSEvents → CFRunLoop pump dispatches → 5 s debounce → Reload → file appears in the shared-files view, matching the Linux/Windows behaviour already verified end-to-end.
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
May 14, 2026
The previous commit's cold-discovery (and the existing amule-project#591 watcher RegisterNewSubdirectory) auto-added new subdirectories to every shared dir indiscriminately. There was no way to express "share /Music but NOT its sensitive nested subfolders" -- the moment a new subdir appeared under any shared root it became part of the share. Desktop users with sensitive nested folders inside a shared root were exposed. This commit introduces a per-root recursive flag, persisted in a new shareddir-recursive.dat file. The watcher and cold-discovery now gate their auto-add behaviour on "ancestor is recursive": only roots the user has explicitly marked recursive via the UI's right- click "share subdirectories recursively" automatically include new/existing subdirectories. Non-recursive shares stay strict. Three on-disk files going forward: * shareddir-recursive.dat (new): roots the user marked recursive. * shareddir-explicit.dat (new): roots the user added explicitly, without recursion. * shareddir.dat (existing): regenerated as the runtime union for backwards compatibility -- older binaries and external scripts that read or write this file see the same effective list as before. `CPreferences::shareddir_list` (the in-memory list the share scan, watcher, etc. read as authoritative) is recomputed at every ReloadSharedFolders as `shareddir_explicit_list ∪ expand(shareddir_ recursive_list)`. The expansion walks each recursive root's subtree (CDirIterator::Dir) and adds every existing descendant directory. Migration: a pre-existing shareddir.dat with no shareddir-{explicit,recursive}.dat is loaded entirely into the explicit list. Existing users keep their path set but stop silently auto-recursing -- they opt back in per-root via the UI. Safe default for desktop users. Reconciliation runs on every Reload (startup, EC reload, UI reload, watcher-debounced reload), not just startup. Diffs the on-disk shareddir.dat against the expected union; entries written externally (Docker entrypoints, sysadmin edits, old-binary writes) are imported into shareddir_explicit_list, and entries the external writer removed are dropped from explicit (entries that came from the recursive expansion are left alone -- the user's recursive intent overrides single-entry external edits). UI side: PrefsUnifiedDlg::CommitSharedDirsWithProgress already extracted explicit vs recursive intents from the DirectoryTreeCtrl (GetSharedDirectories vs GetRecursiveSharedDirectories). The commit path now assigns each to its dedicated Preferences list instead of conflating them into shareddir_list, and the cancel rollback restores all three. Initial dialog state loads from the two intent lists rather than the runtime union (which would otherwise render auto-discovered subdirs as user-selected items). Watcher side: CSharedDirWatcher::RegisterNewSubdirectory and ::ColdDiscoverSubdirs both check CPreferences::IsRecursiveAncestor() before adding. Explicit shares no longer accumulate subdirs at runtime; recursive shares behave as before. Verified on macOS with a three-mode smoke test: * `recursive-root` (in shareddir-recursive.dat) with pre-existing subA/subB: both auto-expand into shareddir.dat at startup; recursive-root/rec-hot created at runtime is auto-added. * `explicit-root` (in shareddir-explicit.dat) with pre-existing subC: subC NOT in shareddir.dat; explicit-root/exp-hot created at runtime NOT added. * `script-added-dir` written to shareddir.dat externally (Docker-entrypoint pattern): imported into shareddir-explicit.dat on next boot, survives subsequent shareddir.dat regenerations. No file format change to shareddir.dat (still one path per line); the new files use the same format. Older binaries that ignore the new files continue to operate against shareddir.dat as the union, with the caveat that recursive intent is lost when an older binary round-trips shareddir.dat (the markers in shareddir-recursive.dat survive untouched because the old binary doesn't read or write that file).
mrjimenez
pushed a commit
that referenced
this pull request
May 14, 2026
The previous commit's cold-discovery (and the existing #591 watcher RegisterNewSubdirectory) auto-added new subdirectories to every shared dir indiscriminately. There was no way to express "share /Music but NOT its sensitive nested subfolders" -- the moment a new subdir appeared under any shared root it became part of the share. Desktop users with sensitive nested folders inside a shared root were exposed. This commit introduces a per-root recursive flag, persisted in a new shareddir-recursive.dat file. The watcher and cold-discovery now gate their auto-add behaviour on "ancestor is recursive": only roots the user has explicitly marked recursive via the UI's right- click "share subdirectories recursively" automatically include new/existing subdirectories. Non-recursive shares stay strict. Three on-disk files going forward: * shareddir-recursive.dat (new): roots the user marked recursive. * shareddir-explicit.dat (new): roots the user added explicitly, without recursion. * shareddir.dat (existing): regenerated as the runtime union for backwards compatibility -- older binaries and external scripts that read or write this file see the same effective list as before. `CPreferences::shareddir_list` (the in-memory list the share scan, watcher, etc. read as authoritative) is recomputed at every ReloadSharedFolders as `shareddir_explicit_list ∪ expand(shareddir_ recursive_list)`. The expansion walks each recursive root's subtree (CDirIterator::Dir) and adds every existing descendant directory. Migration: a pre-existing shareddir.dat with no shareddir-{explicit,recursive}.dat is loaded entirely into the explicit list. Existing users keep their path set but stop silently auto-recursing -- they opt back in per-root via the UI. Safe default for desktop users. Reconciliation runs on every Reload (startup, EC reload, UI reload, watcher-debounced reload), not just startup. Diffs the on-disk shareddir.dat against the expected union; entries written externally (Docker entrypoints, sysadmin edits, old-binary writes) are imported into shareddir_explicit_list, and entries the external writer removed are dropped from explicit (entries that came from the recursive expansion are left alone -- the user's recursive intent overrides single-entry external edits). UI side: PrefsUnifiedDlg::CommitSharedDirsWithProgress already extracted explicit vs recursive intents from the DirectoryTreeCtrl (GetSharedDirectories vs GetRecursiveSharedDirectories). The commit path now assigns each to its dedicated Preferences list instead of conflating them into shareddir_list, and the cancel rollback restores all three. Initial dialog state loads from the two intent lists rather than the runtime union (which would otherwise render auto-discovered subdirs as user-selected items). Watcher side: CSharedDirWatcher::RegisterNewSubdirectory and ::ColdDiscoverSubdirs both check CPreferences::IsRecursiveAncestor() before adding. Explicit shares no longer accumulate subdirs at runtime; recursive shares behave as before. Verified on macOS with a three-mode smoke test: * `recursive-root` (in shareddir-recursive.dat) with pre-existing subA/subB: both auto-expand into shareddir.dat at startup; recursive-root/rec-hot created at runtime is auto-added. * `explicit-root` (in shareddir-explicit.dat) with pre-existing subC: subC NOT in shareddir.dat; explicit-root/exp-hot created at runtime NOT added. * `script-added-dir` written to shareddir.dat externally (Docker-entrypoint pattern): imported into shareddir-explicit.dat on next boot, survives subsequent shareddir.dat regenerations. No file format change to shareddir.dat (still one path per line); the new files use the same format. Older binaries that ignore the new files continue to operate against shareddir.dat as the union, with the caveat that recursive intent is lost when an older binary round-trips shareddir.dat (the markers in shareddir-recursive.dat survive untouched because the old binary doesn't read or write that file).
4 tasks
mrjimenez
pushed a commit
that referenced
this pull request
May 27, 2026
CSharedDirWatcher::RegisterAllPaths() previously subscribed only the
explicit shareddir_list to the wxFileSystemWatcher backend. But
CSharedFileList::Reload() treats three sources as shared (around
SharedFileList.cpp:370):
1) the global Incoming dir (thePrefs::GetIncomingDir())
2) each category's Incoming (theApp->glob_prefs->GetCatPath(i),
i = 1 .. GetCatCount()-1)
3) the explicit shareddir_list
So the auto-rescan introduced in #591 was missing (1) and (2):
* A file dropped into ~/.aMule/Incoming did not trigger the watcher
-> the file never appeared in Shared until some unrelated CREATE
fired elsewhere in the watched tree and incidentally rescanned.
* Same story for any category-specific incoming dir.
Reproducer from #741:
echo 123456 > ~/.aMule/Incoming/file_in_Incoming
# wait — never auto-shared
echo 654321 > ~/.aMule/extra/file_in_extra # (extra is shared)
# both files appear in Shared within seconds
Completed downloads aren't affected: CPartFile signals the scanner
directly when a file transitions to Incoming, so end-of-transfer
sharing keeps working. The bug only bites manual / external drops
into Incoming, which is exactly the case #741 walks through.
Fix: build the effective watch list by mirroring Reload()'s three
sources (Incoming + cat paths + shareddir_list), deduplicating
by GetRaw() so a user who happened to also list Incoming as an
explicit shared dir doesn't get a duplicate watch registration.
The dedup matters on Linux/BSD/Windows (per-dir Add()) and on
macOS (FSEvents rejects overlapping AddTree streams; an explicit
ancestor + Incoming-as-descendant still gets pruned by the existing
covered_by_ancestor logic further down).
Reported by @danim7 (#741).
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automatic rescan of shared directories. aMule currently only rescans shared folders on explicit user action ("Reload shared files") or at daemon start; dropping a file into a shared folder leaves it un-shared until the next manual reload — a long-standing gap for anyone who curates a library outside aMule (rsync, Finder drag-and-drop, torrent client move-on-complete, etc.).
A new
CSharedDirWatcherobserves every path inshareddir_listviawxFileSystemWatcher(inotify on Linux, ReadDirectoryChangesW on Windows, FSEvents on macOS, kqueue on BSD) and triggers a debouncedCSharedFileList::Reload5 s after the last event. The window is long enough to coalesce the bursts produced bytar -x/rsync, short enough that a single drag-and-drop feels responsive.New subdirectories created under any watched parent are auto-appended to
shareddir_listand persisted via a newCPreferences::SaveSharedFolders()that writes onlyshareddir.dat, so the change survives a restart without forcing a fullpreferences.datrewrite on every mkdir event. That mirrors what users already expect from the recursive-share button.The feature is exposed as a new pref
AutoRescanSharedDirs(default ON) with a checkbox in the Directories preferences panel. A new EC tagEC_TAG_DIRECTORIES_AUTO_RESCAN(0x1A05) carries the pref over the EC channel so amulegui can flip it on a remote amuled, with the daemon-side Apply path enabling/disabling the watcher live (no restart needed).Platform-specific details
Linux (inotify) and Windows (ReadDirectoryChangesW). Per-directory
Add().shareddir_listalready enumerates every subdirectory individually when a user uses the recursive-share button, so the inotify watch count tracksshareddir_list.size()rather than total subtree depth. Per-pathAdd()failures (e.g. hitting/proc/sys/fs/inotify/max_user_watches) are logged and skipped; partial coverage is the graceful-degradation path rather than a hard error.macOS (FSEvents). Routes through
AddTree()instead. wx 3.3.2's bareAdd()falls through to the kqueue base and returns false on otherwise-openable directories — verified against wx upstream and reproduced locally. FSEvents is also the API Apple actually recommends for directory monitoring.AddTree()is recursive, so the registration loop skipsshareddir_listentries whose ancestor is also in the list to avoid overlapping streams (wx silently returns false on the second).Deferred watcher construction. wx's inotify backend hard-requires an active wx event loop at
wxFileSystemWatcher::Init()— without one,m_servicestays null and everyAdd()returns false.CamuleApp::OnInit()runs before the loop starts, so the firstEnable()from there queues itself viawxEvtHandler::CallAfterand re-runs on the next loop iteration. Calls from the prefs dialog or EC apply path already run inside an active loop and take the immediate branch.macOS amuled — CFRunLoop pump (second commit). wx's FSEvents wrapper schedules its stream on the calling thread's
CFRunLoop. Under aMule.app the Cocoa main loop spins that runloop, so callbacks deliver normally. Under amuled (wxAppConsole) the loop never spins, so events queue forever. A 200 mswxTimercallsCFRunLoopRunInMode(kCFRunLoopDefaultMode, 0, true)— a non-blocking drain — so queued FSEvents callbacks dispatch into wx's event queue and reach our handler. Gated by!wxTheApp->IsGUI()so aMule.app is unaffected (its Cocoa main loop already pumps).End-to-end verification
amuledamuled.exeamuledaMule.appAll four targets (
amule,amuled,amulecmd,amulegui) compile and link clean on macOS, Linux ARM64, and Windows ARM64.