PartFile: gate periodic SavePartFile on in-memory dirty flag - #671
Merged
mrjimenez merged 2 commits intoMay 22, 2026
Conversation
Follow-up to the amule-project#669 thread (Stoatwblr). The atomic-rename in SavePartFile cuts the per-save cost ~3x, but the periodic save still fired every 60s for every dirty partfile regardless of whether anything had actually changed since the previous write -- an idle sharer with N partfiles produces N byte-identical .part.met writes per minute forever. FlushBuffer's tail-end SavePartFile() now runs only when m_metDirty is set. The flag flips when a field that is serialized to the .part.met mutates: - Gap list (AddGap / FillGap) -- the dominant trigger during active downloads. CPartFileWriteThread's chunk-write completion goes through FillGap, so .part data-file modtime changes are covered transitively. - Status / paused flag (SetStatus, PauseFile, ResumeFile). - Priorities (SetDownPriority, CKnownFile::SetUpPriority, SetAutoDownPriority, CKnownFile::SetAutoUpPriority -- the last two moved out of the header so the partfile case can call MarkMetDirty from the base class). - Category (SetCategory). - Last-seen-complete timestamp. - Filename (SetFileName). - Corrupted-part list and AICH-error transitions are persisted via the existing explicit SavePartFile() callers in the ICH/AICH recovery paths; those clear the flag. A successful SavePartFile() clears m_metDirty. LoadPartFile clears it at each successful return, since the in-memory state matches the freshly-read .met and setter calls during tag parsing should not cause a spurious rewrite on the first tick after load. The destructor's explicit save persists final state on graceful shutdown regardless of the flag, so stat counters (transferred, AllTimeRequests, etc.) -- which deliberately do not mark dirty -- still survive normal shutdowns; they fall behind only on crash, same as the existing behavior for many other in-RAM stats. The 60s BUFFER_TIME_LIMIT gate before FlushBuffer is unchanged -- the flag only changes whether the periodic save fires, not when. Worst case is bounded to one save per partfile per 60s. Best case (truly idle seeder) is zero writes for as long as nothing changes. Direct SavePartFile() callers from state-change paths (priority change, category change, etc.) continue to save immediately.
Closed
Earlier commit on this branch gated the periodic SavePartFile on
m_metDirty. That correctly cuts byte-identical writes on idle
seeders, but undersaves upload-stat counters
(AllTimeRequests / AllTimeAccepts / AllTimeTransferred) on a
seeder with active uploads: stats increment on every served
chunk but were intentionally excluded from m_metDirty (otherwise
the dirty-flag optimisation collapses for any popular file), so
they would persist only at graceful shutdown -- losing the whole
session on crash.
Two-tier model:
- m_metDirty -- hard state (gap list, status, priority, ...)
flips this. Saved at the next FlushBuffer tick (~60 s),
same as before.
- m_statsDirty -- soft stats only. Saved only when the
STATS_HEARTBEAT_MS (10 min) elapses since the last save,
AND nothing else has forced a save in the meantime. A
regular dirty save automatically resets the heartbeat
because m_lastMetSaveTick is updated on every successful
save.
Tagged in CFileStatistic::AddRequest / AddAccepted /
AddTransferred (the three increment paths) with an IsPartFile()
guard so completed files in known.met are unaffected.
Net effect: an idle seeder with no activity writes nothing. A
pure seeder with active uploads writes its .met at most once
per 10 min instead of once per 60 s -- 6x fewer writes for the
same accuracy bound. An active downloader's cadence is
unchanged.
got3nks
marked this pull request as ready for review
May 21, 2026 22:28
ngosang
pushed a commit
to ngosang/amule
that referenced
this pull request
Jul 28, 2026
…ect#180 phase 1) (amule-project#671) * feat(gui): port the Preferences sidebar to wxDataViewCtrl (amule-project#180 phase 1) Phase 1 of the accessibility work in amule-project#180: the Preferences sidebar was a plain wxListCtrl, invisible/unusable to screen readers on macOS (VoiceOver) and Linux (Orca) -- wxDataViewCtrl is native on both those platforms (wxHAS_NATIVE_DATAVIEWCTRL for __WXGTK__/__WXOSX__), falling back to the generic wx-drawn control only on Windows, where the answer was already "works the same as before" per got3nks's review. m_PrefsIcons is now a wxDataViewListCtrl (single icon+text column, no tree mode -- the sidebar is a flat list today, per the earlier correction to the amule-project#180 thread; tree mode is future work for subcategories, not part of this port). Along the way, fixes the positional-index bug flagged in the original amule-project#180 phasing proposal: OnPrefsPageChange used to derive both the current wxPanel* and the pages[] lookup from the row's live *position* in the sidebar (event.GetIndex()), which drifts whenever the Server or IP2Country row is hidden/re-shown -- a comment already noted this had bitten the reset-button-visibility check once. Each row's item data is now the page's stable pages[] array index instead, set once at insertion and never derived from position; m_pageWidgets (indexed by that same stable index) replaces the old SetItemPtrData(widget) lookup. EnableServerTab's insert-at-m_IndexServerTab positional logic is otherwise unchanged -- it was already correct for that purpose, just fragile for the *different* purpose OnPrefsPageChange was (mis)reusing it for. Column width is computed by measuring actual (translated) label text extents rather than relying on wxDataViewColumn's own auto-size timing, which isn't reliably immediate across native vs generic backends. Verified: built and ran both amule and amuleGUI on macOS. Visually confirmed (by a human, not just me, since I can't reliably screenshot this myself) that every page label renders in full (an early version clipped the longest ones -- fixed by widening the padding budget), page switching works, and the Server tab correctly disappears/ reappears at the right position when toggling the ED2K network on and off. Windows and Linux screenshots still needed before this is merge-ready, per got3nks's stated requirement -- see PR description. * fix(gui): guard invalid dataview item and fix sidebar label measurement EVT_DATAVIEW_SELECTION_CHANGED also fires when the selection is cleared, unlike the old EVT_LIST_ITEM_SELECTED, and GetItemData() on an invalid item crashes outright. Also measure the sidebar label width with m_PrefsIcons's own font instead of the dialog's, since the two can differ on native backends. Addresses review feedback from got3nks on amule-project#671. Co-Authored-By: Claude Sonnet 5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5 <[email protected]>
ngosang
pushed a commit
to ngosang/amule
that referenced
this pull request
Jul 28, 2026
…-project#663) * feat(gui): remove the global Connect/Disconnect toolbar button Closes amule-project#402. Per-network controls already exist (Kad pane's Start/Stop, Servers pane's ED2K connect/disconnect), the connection state is already shown in the status bar, and after discussing relocating it into the Networks view the agreed call (got3nks + ngosang) was simply to remove it -- mirrors what amule-project#585 already did on the Web UI side. Removed the toolbar tool, its three skin icons (Toolbar_Connect/ Disconnect/Connecting) and their bitmap wiring, the button-update block in ShowConnectionState(), and the two EnableTool(ID_BUTTONCONNECT, ...) call sites. CamuleDlg::OnBnConnect() itself stays -- CMuleTrayIcon::DoConnectDisconnect() still calls it for the tray icon's own connect/disconnect action, which is unaffected by this change. ShowConnectionState()'s skinChanged parameter was only ever read by the now-removed block, so it's gone too, along with the one call site that passed true and the forceUpdate plumbing that reached it through GuiEvents.cpp's ShowConnState() free function. Left ID_BUTTONCONNECT and muuli_wdr.cpp's muleToolbar() (which still calls it) alone -- that function predates Apply_Toolbar_Skin, is never called anywhere, and touching pre-existing dead code is out of scope for this fix. Verified: built and ran both amule and amulegui cleanly, full unit test suite passing (26/26). Confirmed via the po/ diff that "Connect"/ "Disconnect"/"Cancel" remain in the catalog (still used by other per-network buttons) -- only their now-orphaned toolbar-specific tooltip strings were dropped. Could not get a real on-screen visual confirmation of the toolbar layout -- same macOS Accessibility automation limitations as amule-project#180 got in the way of screenshotting the actual app window. * i18n: regenerate po/ catalogs after rebasing onto current master Mechanical rebase to resolve conflicts against master's amule-project#642/amule-project#643 -- no source changes here. Resolved a real conflict in amuleDlg.cpp's event table: amule-project#642 added the Alt+<letter> EVT_MENU block right where this branch removed the EVT_TOOL(ID_BUTTONCONNECT, ...) line; kept amule-project#642's block, dropped the connect-button line as intended. Regenerated again via scripts/update-po.sh after later rebasing onto master's amule-project#671 (Preferences sidebar) and amule-project#665/amule-project#662 (amuleapi credentials, sparse part-file setting), which had added strings this branch's catalogs didn't have yet. * feat(gui): turn the per-network Disconnect buttons into Connect/Cancel/Disconnect toggles Relocates the removed global toolbar button's functionality into the ED2K (IDC_ED2KDISCONNECT) and Kad (ID_KADDISCONNECT) panes instead of just dropping it, per got3nks's proposal on amule-project#663: each button now mirrors ed2kState/kadState (off/connecting/connected) using the existing connButImg() bitmaps, and can independently connect or cancel/disconnect its own network -- something the old OR-toggled global button never allowed. The ED2K pane also gains a Connect button it never had (previously Disconnect-only). CServerWnd::UpdateED2KConnectButton() / CKadDlg::UpdateConnectButton() are called from CamuleDlg::ShowConnectionState() alongside the existing UpdateED2KInfo()/UpdateKadInfo() calls, and once from each pane's own ctor/Init() for the initial paint. OnBnClickedED2KDisconnect and OnBnClickedDisconnectKad gained the missing "connect when off" branch; OnBnConnect and the tray icon are untouched. Verified interactively on Windows: both buttons show the correct label/icon for their live state, ED2K disconnect/reconnect works from its own button, and disabling ED2K in Preferences while Kad stays connected confirms the two toggle independently. po/ regenerated via scripts/update-po.sh to match the new button labels ("Disconnect Kad" is now unused and moves to the obsolete section; no new translatable strings). * fix(gui): drop the per-network connect-button state cache got3nks flagged two bugs in review (PR amule-project#663): the static cache's early return also skipped Enable(), so the button could get stuck disabled once IsReady()/network-pref state changed without the Connect/Cancel/Disconnect state itself changing (master had two now- deleted escape hatches -- the skinChanged force-path and an unconditional EnableTool call -- that used to paper over this). Worse, the cache was function-scope static, surviving widget recreation, so a fresh pane's Init() call could early-return and leave the wxDesigner default label ("Disconnect Kad") on screen. Dropping the cache removes both: SetLabel/SetBitmap/Enable are cheap on a plain wxButton (unlike the old toolbar tool, which had real native-image-list reasons to memoize, see amuleDlg.cpp:1054-ish), and master's own second call site already did an unconditional EnableTool every tick without issue. Second review round (macOS + Ubuntu ARM64, monolithic + amuleGUI) confirmed the toggle works functionally but flagged three cosmetic issues, all fixed here: - No gap between the connect-button icon and its label. The obvious fix, wxButton::SetBitmapMargins(), is NOT portable (wxOSX overrides it, wxGTK inherits the base no-op) -- prefixing the label with a literal space outside the _() call behaves identically on both and introduces no new msgid. - The Kad button was stretched full-width by its sizer's .Expand() flag, while the ED2K one sizes naturally; GTK and macOS then filled the slack differently (centered vs. icon-left). Dropped .Expand() so the two buttons size the same way -- trades away lining up with the "Bootstrap from known clients" button above it, but the two connect buttons reading as the same control matters more. - CServerWnd::UpdateED2KConnectButton() and CKadDlg::UpdateConnectButton() were near-identical (same 3-state enum, same switch, same three bitmaps). Factored into a shared SetConnectButtonState(button, state, enabled) helper in muuli_wdr.cpp/.h, next to connButImg() which it reuses -- so the icon-spacing fix above lives in one place instead of two. Verified interactively on Windows: re-tested full disconnect/reconnect cycle on both panes, toggling ED2K off and back on via Preferences (existing behavior: needs an app restart to re-enable) with correct state on the fresh post-restart paint, and confirmed by screenshot that both buttons now show a visible icon/label gap and size the same way (Kad no longer stretched). Also confirmed amulegui (CLIENT_GUI) builds clean with these changes; did not perform full interactive remote-daemon testing. * fix(gui): first-paint layout bug + connect-button placement, per review Two more issues from got3nks's Linux/GTK pass on amule-project#663. 1. Bug: the ED2K connect button painted with the icon overlapping the label on first show (Kad was fine, only by luck of timing). CServerWnd::UpdateED2KConnectButton() runs after sizer->Show(this, TRUE) in the ctor, so SetBitmap() grows the button's best size after the row was already measured against the text-only label, and nothing re-lays it out until a window resize. Fixed by calling Layout() on the button's parent at the end of the shared SetConnectButtonState() helper -- covers both panes, and stops Kad from relying on being laid out late (its notebook page is only measured once shown, after Init() already set its bitmap). 2. Layout: moved both connect/disconnect toggles to lead their pane instead of sitting among secondary controls, per got3nks's request: - ED2K (serverListDlgUp): toggle now on its own top row, right- aligned via a stretch spacer, directly above the server list. The "Add server manually" form (name/IP/port/Add) moved below the table instead of sharing a row with the toggle; the vertical separator that used to sit between them is gone (no longer needed once they're not adjacent). - Kad (KadDlg): toggle moved out of the "Bootstrap" static box (where it sat under "Bootstrap from known clients") to its own top row above the box, right-aligned the same way as ED2K's. item0 changed from a 1x1 FlexGridSizer (which only ever held the two-column area) to a plain vertical wxBoxSizer so it can hold the new top row plus the existing two-column layout. Both panes now read "primary control on top, secondary/manual controls below" -- matching shape on both tabs. Verified interactively on Windows: screenshotted both panes, confirmed no icon/label overlap on first paint (no window resize needed), and re-ran the disconnect/reconnect toggle cycle on both after the layout change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #670 (issue #669). #670 cut the per-save cost ~3x by switching
SavePartFilefrom copy-rewrite-copy to atomic-rename. This PR closes the remaining hole: the periodic 60-second save inFlushBufferstill fired for every partfile regardless of whether anything had actually changed since the previous write.What
A sharer with N partfiles produces N
.part.metwrites per minute even when nothing has changed (no new chunks, no new sources, no state transitions). For an idle seeder that is wasted disk work: every cycle writes byte-identical content. On SSDs and COW filesystems (ZFS, btrfs) this is meaningfully more expensive than reads, and compounds with the volume.How
Two-tier dirty model on
CPartFile:Hard state —
m_metDirty. Flipped on every mutation of a field whose change should land on disk promptly:AddGap/FillGap(both overloads).CPartFileWriteThread's chunk-write completion goes throughFillGap, so.partdata-file modtime changes are covered transitively.SetStatus,PauseFile,ResumeFile.SetDownPriority,CKnownFile::SetUpPriority,SetAutoDownPriority,CKnownFile::SetAutoUpPriority(the last two moved out of the header so the partfile case can callMarkMetDirtyfrom the base class).SetCategory.SetFileName.SavePartFile()calls in the ICH/AICH recovery paths.FlushBuffer's tail-end save fires at the next 60-s tick whenm_metDirtyis set.Soft stats —
m_statsDirty. Flipped on every upload-counter increment (CFileStatistic::AddRequest/AddAccepted/AddTransferred). These increment on every served chunk so they're deliberately excluded fromm_metDirty— otherwise any popular file would re-dirty its partfile constantly and the optimisation collapses. Instead they're persisted on a 10-minute heartbeat measured from the last successful save. Because every successful save resetsm_lastMetSaveTick, a regular dirty save automatically defers the next stats save by 10 more minutes — no extra writes when the two coincide.A successful
SavePartFile()clears both dirty bits and stampsm_lastMetSaveTick.LoadPartFileclears both at each successful return and seeds the tick from load time, so a freshly-loaded partfile (whose in-memory state matches the just-read.met) doesn't rewrite identical content on the first tick. The destructor's explicit save persists final state on graceful shutdown regardless of either flag.Cadence
The 60-second
BUFFER_TIME_LIMITgate beforeFlushBufferis untouched. The flags only change whether the periodic save fires, not when:m_metDirty); a coinciding stats heartbeat is absorbedWorst case is bounded to one save per partfile per 60 s. Best case (truly idle, no uploads) is zero writes until shutdown. Stat counters lose at most 10 min on crash, vs the previous 60 s — accepted trade for ~83% fewer writes on a busy sharer.
Status
Draft — awaiting OP confirmation on #669 before promoting. Builds clean on macOS (monolithic + remotegui + daemon).