kad: revert popularity decay; keep GetCommonFileName all-zero robustness - #805
Merged
mrjimenez merged 1 commit intoJun 1, 2026
Conversation
Reverts the decay tick + zero-popularity escape hatch landed in df10aa8 ("kad: add popularity decay so saturated m_filenames can still rotate") and addressed in PR amule-project#799. Discussion on amule-project#795 with @irwir after amule-project#799 landed surfaced the existing entry-lifetime cleanup that already handles the "publisher offline => entry invalidation" case the decay was targeting: - CKeyEntry::CleanUpTrackedPublishers ages out individual publisher IPs from m_publishingIPs after KADEMLIAREPUBLISHTIMEK (24 h) of silence. - CIndexed::Clean deletes the whole CKeyEntry when its m_tLifeTime expires (same 24 h horizon, extended on every publisher refresh). So "publisher went offline" is handled at both the per-IP and per-entry granularities. The only thing left uncleaned is a single name variant within an active CKeyEntry, which is exclusively a saturated-m_filenames concern. Empirical data from a live HighID production node (~4000 merges over ~4 h across ~1800 indexed keys) shows the saturated-key count never lifts off zero -- so the regime decay was designed for doesn't appear in practice on a healthy node. At the rare saturated cases the "freeze at the cap" behaviour the decay was overriding is now the correct behaviour: bounded RAM, no spurious rotation of legitimate vote winners when traffic dips. The 100-entry cap from amule-project#795 / amule-project#314 stays. It protects against a different case from the whole-entry TTL: an *active* CKeyEntry that adversarial or pathological publishers feed unbounded variant names under the same hash. As @irwir noted in this thread, "setting a limit would be reasonable to prevent attacks and misuse." The cap is a single integer comparison on insert and lines up with the m_publishingIPs cap right next to it. Concretely reverted: - CKeyEntry::MERGES_PER_DECAY_TICK static + m_mergeCounter member. - The decay tick block at the end of MergeIPsAndFilenames. - The "weakest->m_popularityIndex == 0" escape hatch in pushBounded -- back to the original strict `>` comparison from amule-project#795. Deliberately kept: the GetCommonFileName robustness fix from df10aa8. Our own code no longer produces popularity-0 entries after this revert, but two paths we don't control can still land them in m_filenames: on-disk data from any node that ran the decay build (it could have written 0-popularity entries to known2.met before this revert lands), and malformed/adversarial publishes that send popularity = 0 over the wire. The picker's seed-from-first / "pick first on tie" fallback closes a sharp edge in a function whose output is on the wire (TAG_FILENAME, SearchTermsMatch, CIndexed::AddKeyword reject path) for essentially no cost. Refs amule-project#795.
2 tasks
1 task
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 4, 2026
…ndex Adds 55+ merged PRs to the 3.0.0 changelog since the last update (amule-project#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796 (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths). - Bug Fixes & Stability: post-amule-project#744 fixes including EC notification leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728), amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760), PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788, amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756), Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833, amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774). - Translations: late-cycle wave covering French/Turkish manpages (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for date+version drift (amule-project#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 4, 2026
…ndex Adds 55+ merged PRs to the 3.0.0 changelog since the last update (amule-project#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796 (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths). - Bug Fixes & Stability: post-amule-project#744 fixes including EC notification leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728), amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760), PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788, amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756), Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833, amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774). - Translations: late-cycle wave covering French/Turkish manpages (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for date+version drift (amule-project#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
mrjimenez
pushed a commit
that referenced
this pull request
Jun 4, 2026
Adds 55+ merged PRs to the 3.0.0 changelog since the last update (#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for #785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), #794 (.dmg amuleweb path), #789 (<OS>-<arch> artifact naming), #780 / #796 (Windows DPI + comctl32 manifest), #784 (FHS share/amule paths). - Bug Fixes & Stability: post-#744 fixes including EC notification leak (#797), big-library scaling (#736, #840 superseding #728), amulegui ghost entries (#810, #819, #841, #824, #830, #760), PartFile early hash (#762), server protocol fixes (#835, #788, #721, #787), crypto stream UB (#779), UAF prevention (#756), Kad rotation (#795, #799/#805), GTK warning silencing (#833, #826/#836), and the clang-tidy worklist (#770, #772-#774). - Translations: late-cycle wave covering French/Turkish manpages (#753/#754/#776), Galician (#763), Slovenian (#771), pt-BR (#768/#775/#812), French (#811), plus man-page tooling for date+version drift (#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (#817/#818/#821/#828/#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with #746-#845 + #841.
mrjimenez
pushed a commit
to mrjimenez/amule
that referenced
this pull request
Aug 8, 2026
…olumn sizing (amule-project#805) Restores behaviour the wxDataViewCtrl port left behind, and fixes a column-sizing bug that came with it. Column show/hide menu. Right-clicking a header opened a check-menu of columns (CMuleListCtrl::OnColumnRClick/OnMenuSelected); the ported search list bound no header right-click handler, so columns could not be hidden at all. Restored with the original semantics: hiding caches the width so re-showing restores it, and the state persists through CListColumnStore's existing width handling rather than a second mechanism. COL_SIZE_MIN moves from a file-local constant in MuleListCtrl.cpp to ListColumnStore.h so both lists read one threshold. Hidden state is tracked by CSearchListCtrl rather than read back from the control: a hidden wxDataViewColumn keeps reporting its previous width, so width alone cannot answer whether a column is hidden, and that question is asked by the menu's check marks, the expander's leftmost-visible search, the persisted width and the cross-tab sync. SyncLists() carries the hidden state with the width, since copying width alone left other tabs showing a column this one had hidden. The group expander follows the leftmost visible column, including after LoadColumnSettings(), so hiding the column carrying it does not strand the triangles or leave children unreachable. Trailing column collapse. macOS gives leftover space to the last resizable column, so with the columns wider than the control the trailing one was sized to nothing and GetWidth() returned zero. CListColumnStore reads a width <= 0 as hidden and persists it negative, so each shutdown marked that column hidden and the next launch repeated it on whichever column had become last, losing one per restart. An empty, resizable spacer column now occupies that position; RealColumnCount() keeps it out of the menu, the persisted widths, the hidden-state tracking and the cross-tab sync, and the width adapter will not pass a zero for a column that is not hidden. This predates the header menu and is specific to the port -- the lists still on CMuleListCtrl persist positive widths for wider column sets. Selection keys on macOS. shift+page-up/down extends the selection on GTK and MSW, whose backends implement it, and did nothing on macOS, where NSOutlineView moves the view without touching the selection; shift+home/end were unhandled entirely. Both are now handled there so the three ports agree. Unshifted page-up/down still belongs to the platform, which scrolls without moving the selection by convention. Row ordering for both the existing type-ahead search and the new key handling goes through BuildDisplayOrder(), which runs the rows through this list's own comparator -- GetItemByRow()/GetRowByItem() exist only in wx's generic implementation -- and includes the children of expanded groups, since a page is counted in screen rows. Also removes CSearchListCtrl::RemoveResult() and CSearchListModel::NotifyFileRemoved(), dead since the port. Verified interactively on macOS, Ubuntu 26.04 arm64 (wxGTK 3.2.9) and Windows 11 arm64; amule and amulegui built separately on each.
mrjimenez
pushed a commit
to mrjimenez/amule
that referenced
this pull request
Aug 8, 2026
amule-project#796 ported the search list to wxDataViewCtrl and amule-project#805 fixed what that port left behind. amule-project#801 asked whether the remaining lists should each be ported independently or share a base, and deferred the answer until the shape was clear from more than one list. It now is. CMuleDataViewCtrl is the wxDataViewCtrl counterpart of CMuleListCtrl: column widths and their persistence, the header show/hide menu, hidden state, the multi-column sort chain, type-to-select, Cmd/Ctrl+A, the macOS shifted page/home/end keys, the trailing-column spacer, and drag-resize detection. It owns no data; a list supplies its rows through GetDisplayOrder(), their label through GetRowLabel(), and how two of them compare through CompareByColumn(). CMuleVirtualDataViewCtrl is the counterpart of CMuleVirtualListCtrl, and virtual in the same sense: rows are addressed by index through a wxDataViewIndexListModel and nothing is materialised per row. It carries the item-identity bookkeeping an identity-addressed port avoids -- a wxDataViewItem from a row-addressed model encodes the row number, so a deletion silently retargets any item held across it, which is why everything here speaks in wxUIntPtr and re-resolves selection after each mutation. Also the legacy filter API, live re-sort coalesced through one CallAfter and deferred while the user is interacting, bulk append and batch removal, and icon columns. CSearchListCtrl moves onto the plain base, losing 499 lines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts the decay piece landed in #799 (commit
df10aa877) and keeps theGetCommonFileNamerobustness fix that rode in with it. Discussion in #795 with @irwir.Why revert
The decay was solving a problem that's already handled at a coarser granularity by existing cleanup machinery, and the saturation regime it targets doesn't occur in practice.
Existing cleanup already covers "publisher went offline":
CKeyEntry::CleanUpTrackedPublishers(Entry.cpp:543) ages out individual publisher IPs fromm_publishingIPsafterKADEMLIAREPUBLISHTIMEK(24 h) of silence.CIndexed::Clean(Indexed.cpp:379) deletes the wholeCKeyEntrywhen itsm_tLifeTimeexpires (24 h horizon, extended on every publisher refresh).The only case left uncleaned is a single name variant within an active
CKeyEntry, which only matters oncem_filenamessaturates at the 100-entry cap. Empirical data from a live HighID production node (~4000 merges over ~4 h across ~1800 keys) shows the saturated-key count never lifted off zero — so the regime decay was sized for doesn't materialise on a healthy node. At the rare saturated cases the "freeze at the cap" behaviour decay was overriding is the right behaviour: bounded RAM, no spurious rotation of legitimate vote winners when publishing tempo dips.Why the 100-entry cap stays
The decay goes; the cap from #795 / #314 doesn't. They protect against different things:
m_tLifeTime+Clean()) only kicks in when all publishers go silent — an activeCKeyEntrystays alive forever.m_filenamesinside such an active entry would still grow unbounded as adversarial or pathological publishers feed it new variant names under the same hash — straight RAM-bloat / spam vector. As @irwir noted earlier in this thread, "setting a limit would be reasonable to prevent attacks and misuse".Cap is cheap (single integer comparison on insert), bounded, and lines up with the
m_publishingIPscap right next to it. Keeping it.What's reverted
CKeyEntry::MERGES_PER_DECAY_TICKstatic +m_mergeCountermember.MergeIPsAndFilenames.weakest->m_popularityIndex == 0escape hatch inpushBounded— back to the original strict>from kad: replace sort+resize filename cap with O(N) compare-and-skip insertion #795.What's kept and why
The
GetCommonFileNamerobustness fix stays. Our own code no longer produces popularity-0 entries after this revert (SetFileNamecreates at 1, merge only++s, no decay), but two paths we don't control can still land them inm_filenames:known2.metmay carry zero-popularity entries that get loaded back at Entry.cpp:701 via uncheckedReadUInt32. Without the picker fix, those keys would have ~24 h of silent breakage (emptyTAG_FILENAME, falseSearchTermsMatch,Indexed::AddKeywordreject path) until aged out.m_popularityIndexas a rawuint32, a peer can send 0.Both are cheap edges to close in a function whose output is on the wire, regardless of whether decay is in the codebase.
Net diff:
+11 / −71(8 files lighter than before).Refs #795. cc @irwir.