fix: amulegui ghost entries when EC alive-marker arrives for unknown ID - #810
Merged
mrjimenez merged 1 commit intoJun 1, 2026
Merged
Conversation
The EC skip-unchanged opt-in (93a0676, "EC skip-unchanged (4/5)") added a backward-compat alive-marker tag — an EC_TAG_KNOWNFILE or EC_TAG_PARTFILE carrying only the ECID with no child tags — used to signal "this file still exists" without resending its full payload. CKnownFilesRem::ProcessUpdate gates the existing-item update branch on tag->HasChildTags() and correctly skips alive-markers there. The new-item branch did not gate, so if a marker arrived for an ID the client hadn't seen yet (race during bulk Reload via amulecmd, server diff baseline briefly out of step, etc.), it constructed a fresh CKnownFile/CPartFile from an empty tag and got default-zero everything: no name, zero size. amulecmd ("show shared") rendered the files correctly because the server-side state was unaffected; amulegui showed ghost entries that only cleared on restart (which forces m_initialUpdate=true and full-tag resync). Skip alive-markers in the new-item branch and emit a debug log line at logEC so future occurrences are observable instead of silently producing ghosts. A subsequent full-tag update introduces the file normally; the 5-minute backstop guarantees the recovery cycle is bounded. Fixes amule-project#808 (Stoatwblr's first point: "0 byte unnamed file in each of the new directories" after bulk reload).
This was referenced Jun 2, 2026
mrjimenez
pushed a commit
that referenced
this pull request
Jun 2, 2026
…ng metadata was suppressed (#808 follow-up) #810 plugged the first variant of the #808 ghost-entry pattern -- an EC_TAG_KNOWNFILE alive-marker tag with no children arriving for a file ID amulegui has no record of. A second variant with identical symptoms remains. CEC_SharedFile_Tag (ECSpecialCoreTags.cpp:221) runs each metadata field through a per-connection CValueMap (ECSpecialTags.h:59), which suppresses tags whose value hasn't changed since the last cached send. On EC_DETAIL_INC_UPDATE for a file ID amulegui hasn't seen yet, the server may already have cached + suppressed EC_TAG_PARTFILE_HASH / _NAME / _SIZE_FULL -- so the tag still carries plenty of statistical children (request count, accept count, transferred bytes, AICH masterhash, priority...) and the HasChildTags() guard added in #810 lets it through, but CKnownFile(tag) then reads empty hash + empty name + zero size via GetTagByNameSafe and produces the same "0-byte unnamed" ghost entry Stoatwblr reported. Defensive check on the new-item path in CKnownFilesRem::Process Update: if the tag is missing EC_TAG_PARTFILE_HASH (the canonical identity tag), log on logEC and skip. The next full-state poll or the existing entry-mismatch recovery path picks the file up correctly once the server-side cache invalidation puts the identifying metadata back on the wire. Triggered in the wild by Stoatwblr's bulk `updatesharedir` / `reload shared` script: amuled adds files to its shareset, the INC_UPDATE diff to amulegui includes them as new entries, but the per-(connection, ECID) value-map cache may already have suppressed the identity fields, so the ghost entry pattern fires once per affected file. Refs #808 (#810 covered the alive-marker variant of the same bug).
1 task
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 4, 2026
…ndex Adds 55+ merged PRs to the 3.0.0 changelog since the last update (amule-project#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796 (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths). - Bug Fixes & Stability: post-amule-project#744 fixes including EC notification leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728), amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760), PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788, amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756), Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833, amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774). - Translations: late-cycle wave covering French/Turkish manpages (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for date+version drift (amule-project#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 4, 2026
…ndex Adds 55+ merged PRs to the 3.0.0 changelog since the last update (amule-project#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796 (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths). - Bug Fixes & Stability: post-amule-project#744 fixes including EC notification leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728), amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760), PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788, amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756), Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833, amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774). - Translations: late-cycle wave covering French/Turkish manpages (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for date+version drift (amule-project#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
mrjimenez
pushed a commit
that referenced
this pull request
Jun 4, 2026
Adds 55+ merged PRs to the 3.0.0 changelog since the last update (#747, 2026-05-27). Narrative additions cover: - Packaging: expanded the top list to include the macOS per-arch .app bundles and the Windows NSIS installer alongside the existing AppImage / Flatpak / .dmg / .zip entries. New bullets for #785 (alc/alcc/cas/wxcas everywhere + Windows amuleweb), #794 (.dmg amuleweb path), #789 (<OS>-<arch> artifact naming), #780 / #796 (Windows DPI + comctl32 manifest), #784 (FHS share/amule paths). - Bug Fixes & Stability: post-#744 fixes including EC notification leak (#797), big-library scaling (#736, #840 superseding #728), amulegui ghost entries (#810, #819, #841, #824, #830, #760), PartFile early hash (#762), server protocol fixes (#835, #788, #721, #787), crypto stream UB (#779), UAF prevention (#756), Kad rotation (#795, #799/#805), GTK warning silencing (#833, #826/#836), and the clang-tidy worklist (#770, #772-#774). - Translations: late-cycle wave covering French/Turkish manpages (#753/#754/#776), Galician (#763), Slovenian (#771), pt-BR (#768/#775/#812), French (#811), plus man-page tooling for date+version drift (#802). - Contributors: added ngosang for UX feedback on the late-3.0 cycle (#817/#818/#821/#828/#844) and ongoing work on the user-facing manual at amule-org.github.io. - Merged PRs flat index: extended with #746-#845 + #841.
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)") gates the per-file emit on `m_ecGen <= ec_threshold` in three handlers. For unchanged files the server either emits a childless alive-marker tag (legacy clients) or skips silently (partial-update clients). Both shortcuts assume the client has previously received the ECID with full detail. If it hasn't, the alive marker constructs a ghost CKnownFile from an empty tag (amule-project#810 surface) and the silent skip leaves the file invisible. The CValueMap-driven identity suppression on subsequent emissions extends the surface to INC_UPDATE tags that have children but no identifying fields (amule-project#819 surface). The amulegui-side guards in amule-project#810 and amule-project#819 cover that client specifically. Other legacy EC consumers — amuleweb, amulecmd, and third-party clients — have no analogous guard and silently lose newly-shared files until restart. Fix at the source: track per-connection, per-handler ECIDs that have been sent with full detail. Gate the alive-marker / silent- skip shortcut on membership in that set. Files the client hasn't seen fall through to full-detail emission unconditionally, and the emission populates the per-connection CObjTagMap (so subsequent INC_UPDATE diffs work) and inserts into the new sent-with-detail set. Three handlers, three sets — one per request path because each handler has its own `m_lastEcGenSeen*` cadence. Memory cost: bounded by alive-file ECID count per connection (~360 KB at 91k files; insignificant). Never pruned — re-share bumps m_ecGen via SetFilePath / Init so any resurrected ECID naturally takes the full-detail branch again. The amulegui guards from amule-project#810 and amule-project#819 stay as defensive belt-and- suspenders for amulegui talking to unpatched / older daemons.
got3nks
added a commit
to got3nks/amule
that referenced
this pull request
Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)") gates the per-file emit on `m_ecGen <= ec_threshold` in three handlers. For unchanged files the server either emits a childless alive-marker tag (legacy clients) or skips silently (partial-update clients). Both shortcuts assume the client has previously received the ECID with full detail. If it hasn't, the alive marker constructs a ghost CKnownFile from an empty tag (amule-project#810 surface) and the silent skip leaves the file invisible. The CValueMap-driven identity suppression on subsequent emissions extends the surface to INC_UPDATE tags that have children but no identifying fields (amule-project#819 surface). The amulegui-side guards in amule-project#810 and amule-project#819 cover that client specifically. Other legacy EC consumers — amuleweb, amulecmd, and third-party clients — have no analogous guard and silently lose newly-shared files until restart. Fix at the source: track per-connection, per-handler ECIDs that have been sent with full detail. Gate the alive-marker / silent- skip shortcut on membership in that set. Files the client hasn't seen fall through to full-detail emission unconditionally, and the emission populates the per-connection CObjTagMap (so subsequent INC_UPDATE diffs work) and inserts into the new sent-with-detail set. Three handlers, three sets — one per request path because each handler has its own `m_lastEcGenSeen*` cadence. Memory cost: bounded by alive-file ECID count per connection (~360 KB at 91k files; insignificant). Never pruned — re-share bumps m_ecGen via SetFilePath / Init so any resurrected ECID naturally takes the full-detail branch again. The amulegui guards from amule-project#810 and amule-project#819 stay as defensive belt-and- suspenders for amulegui talking to unpatched / older daemons.
mrjimenez
pushed a commit
that referenced
this pull request
Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)") gates the per-file emit on `m_ecGen <= ec_threshold` in three handlers. For unchanged files the server either emits a childless alive-marker tag (legacy clients) or skips silently (partial-update clients). Both shortcuts assume the client has previously received the ECID with full detail. If it hasn't, the alive marker constructs a ghost CKnownFile from an empty tag (#810 surface) and the silent skip leaves the file invisible. The CValueMap-driven identity suppression on subsequent emissions extends the surface to INC_UPDATE tags that have children but no identifying fields (#819 surface). The amulegui-side guards in #810 and #819 cover that client specifically. Other legacy EC consumers — amuleweb, amulecmd, and third-party clients — have no analogous guard and silently lose newly-shared files until restart. Fix at the source: track per-connection, per-handler ECIDs that have been sent with full detail. Gate the alive-marker / silent- skip shortcut on membership in that set. Files the client hasn't seen fall through to full-detail emission unconditionally, and the emission populates the per-connection CObjTagMap (so subsequent INC_UPDATE diffs work) and inserts into the new sent-with-detail set. Three handlers, three sets — one per request path because each handler has its own `m_lastEcGenSeen*` cadence. Memory cost: bounded by alive-file ECID count per connection (~360 KB at 91k files; insignificant). Never pruned — re-share bumps m_ecGen via SetFilePath / Init so any resurrected ECID naturally takes the full-detail branch again. The amulegui guards from #810 and #819 stay as defensive belt-and- suspenders for amulegui talking to unpatched / older daemons.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
amuleguirendered 0-byte unnamed entries for newly-shared files in the Shared Files tab whileamulecmd show sharedrendered the same files correctly (real size + hash). Restartingamuleguicleared the ghosts.Cause
The EC skip-unchanged opt-in landed in
93a067628("EC skip-unchanged (4/5)"). It introduced a backward-compat alive-marker tag — anEC_TAG_KNOWNFILEorEC_TAG_PARTFILEcarrying only the ECID with no child tags — used to signal "this file still exists" without resending its full payload.CKnownFilesRem::ProcessUpdategates the existing-item update branch ontag->HasChildTags()and correctly skips alive-markers there. The new-item branch did not gate. When a marker arrived for an ID the client hadn't seen yet (race during a bulkreload sharedvia amulecmd, server diff baseline briefly out of step, etc.), it constructed a freshCKnownFile/CPartFilefrom an empty tag — yielding default-zero everything: no name, zero size.amulecmdwas unaffected because the server-side state was correct.amuleguirestart fixed it because it forcesm_initialUpdate = trueand the next sync sends full tags for every entry.Fix
Skip alive-markers in the new-item branch and emit a debug log line at
logECso future occurrences are observable. A subsequent full-tag update introduces the file normally; the 5-minute backstop guarantees recovery is bounded if the same ECID never gets re-sent with children.Refs
Fixes #808 (Stoatwblr's first point: "0 byte unnamed file in each of the new directories" after bulk
reload shared).