Skip to content

fix: amulegui ghost entries when EC alive-marker arrives for unknown ID - #810

Merged
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/ec-alive-marker-ghost-entries
Jun 1, 2026
Merged

fix: amulegui ghost entries when EC alive-marker arrives for unknown ID#810
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/ec-alive-marker-ghost-entries

Conversation

@got3nks

@got3nks got3nks commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

amulegui rendered 0-byte unnamed entries for newly-shared files in the Shared Files tab while amulecmd show shared rendered the same files correctly (real size + hash). Restarting amulegui cleared the ghosts.

Cause

The EC skip-unchanged opt-in landed in 93a067628 ("EC skip-unchanged (4/5)"). It introduced a backward-compat alive-marker tag — an EC_TAG_KNOWNFILE or EC_TAG_PARTFILE carrying only the ECID with no child tags — used to signal "this file still exists" without resending its full payload.

CKnownFilesRem::ProcessUpdate gates the existing-item update branch on tag->HasChildTags() and correctly skips alive-markers there. The new-item branch did not gate. When a marker arrived for an ID the client hadn't seen yet (race during a bulk reload shared via amulecmd, server diff baseline briefly out of step, etc.), it constructed a fresh CKnownFile/CPartFile from an empty tag — yielding default-zero everything: no name, zero size.

amulecmd was unaffected because the server-side state was correct. amulegui restart fixed it because it forces m_initialUpdate = true and the next sync sends full tags for every entry.

Fix

Skip alive-markers in the new-item branch and emit a debug log line at logEC so future occurrences are observable. A subsequent full-tag update introduces the file normally; the 5-minute backstop guarantees recovery is bounded if the same ECID never gets re-sent with children.

Refs

Fixes #808 (Stoatwblr's first point: "0 byte unnamed file in each of the new directories" after bulk reload shared).

The EC skip-unchanged opt-in (93a0676, "EC skip-unchanged (4/5)")
added a backward-compat alive-marker tag — an EC_TAG_KNOWNFILE or
EC_TAG_PARTFILE carrying only the ECID with no child tags — used to
signal "this file still exists" without resending its full payload.

CKnownFilesRem::ProcessUpdate gates the existing-item update branch on
tag->HasChildTags() and correctly skips alive-markers there. The
new-item branch did not gate, so if a marker arrived for an ID the
client hadn't seen yet (race during bulk Reload via amulecmd, server
diff baseline briefly out of step, etc.), it constructed a fresh
CKnownFile/CPartFile from an empty tag and got default-zero
everything: no name, zero size. amulecmd ("show shared") rendered the
files correctly because the server-side state was unaffected;
amulegui showed ghost entries that only cleared on restart (which
forces m_initialUpdate=true and full-tag resync).

Skip alive-markers in the new-item branch and emit a debug log line
at logEC so future occurrences are observable instead of silently
producing ghosts. A subsequent full-tag update introduces the file
normally; the 5-minute backstop guarantees the recovery cycle is
bounded.

Fixes amule-project#808 (Stoatwblr's first point: "0 byte
unnamed file in each of the new directories" after bulk reload).
@mrjimenez
mrjimenez merged commit 9113df3 into amule-project:master Jun 1, 2026
7 checks passed
mrjimenez pushed a commit that referenced this pull request Jun 2, 2026
…ng metadata was suppressed (#808 follow-up)

#810 plugged the first variant of the #808 ghost-entry pattern --
an EC_TAG_KNOWNFILE alive-marker tag with no children arriving for
a file ID amulegui has no record of.

A second variant with identical symptoms remains.
CEC_SharedFile_Tag (ECSpecialCoreTags.cpp:221) runs each metadata
field through a per-connection CValueMap (ECSpecialTags.h:59),
which suppresses tags whose value hasn't changed since the last
cached send. On EC_DETAIL_INC_UPDATE for a file ID amulegui hasn't
seen yet, the server may already have cached + suppressed
EC_TAG_PARTFILE_HASH / _NAME / _SIZE_FULL -- so the tag still
carries plenty of statistical children (request count, accept
count, transferred bytes, AICH masterhash, priority...) and the
HasChildTags() guard added in #810 lets it through, but
CKnownFile(tag) then reads empty hash + empty name + zero size via
GetTagByNameSafe and produces the same "0-byte unnamed" ghost
entry Stoatwblr reported.

Defensive check on the new-item path in CKnownFilesRem::Process
Update: if the tag is missing EC_TAG_PARTFILE_HASH (the canonical
identity tag), log on logEC and skip. The next full-state poll or
the existing entry-mismatch recovery path picks the file up
correctly once the server-side cache invalidation puts the
identifying metadata back on the wire.

Triggered in the wild by Stoatwblr's bulk `updatesharedir` /
`reload shared` script: amuled adds files to its shareset, the
INC_UPDATE diff to amulegui includes them as new entries, but
the per-(connection, ECID) value-map cache may already have
suppressed the identity fields, so the ghost entry pattern fires
once per affected file.

Refs #808 (#810 covered the alive-marker variant of the same bug).
@got3nks
got3nks deleted the fix/ec-alive-marker-ghost-entries branch June 3, 2026 14:16
got3nks added a commit to got3nks/amule that referenced this pull request Jun 4, 2026
…ndex

Adds 55+ merged PRs to the 3.0.0 changelog since the last update
(amule-project#747, 2026-05-27). Narrative additions cover:

- Packaging: expanded the top list to include the macOS per-arch .app
  bundles and the Windows NSIS installer alongside the existing
  AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785
  (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg
  amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796
  (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths).

- Bug Fixes & Stability: post-amule-project#744 fixes including EC notification
  leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728),
  amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760),
  PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788,
  amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756),
  Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833,
  amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774).

- Translations: late-cycle wave covering French/Turkish manpages
  (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR
  (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for
  date+version drift (amule-project#802).

- Contributors: added ngosang for UX feedback on the late-3.0
  cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the
  user-facing manual at amule-org.github.io.

- Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
got3nks added a commit to got3nks/amule that referenced this pull request Jun 4, 2026
…ndex

Adds 55+ merged PRs to the 3.0.0 changelog since the last update
(amule-project#747, 2026-05-27). Narrative additions cover:

- Packaging: expanded the top list to include the macOS per-arch .app
  bundles and the Windows NSIS installer alongside the existing
  AppImage / Flatpak / .dmg / .zip entries. New bullets for amule-project#785
  (alc/alcc/cas/wxcas everywhere + Windows amuleweb), amule-project#794 (.dmg
  amuleweb path), amule-project#789 (<OS>-<arch> artifact naming), amule-project#780 / amule-project#796
  (Windows DPI + comctl32 manifest), amule-project#784 (FHS share/amule paths).

- Bug Fixes & Stability: post-amule-project#744 fixes including EC notification
  leak (amule-project#797), big-library scaling (amule-project#736, amule-project#840 superseding amule-project#728),
  amulegui ghost entries (amule-project#810, amule-project#819, amule-project#841, amule-project#824, amule-project#830, amule-project#760),
  PartFile early hash (amule-project#762), server protocol fixes (amule-project#835, amule-project#788,
  amule-project#721, amule-project#787), crypto stream UB (amule-project#779), UAF prevention (amule-project#756),
  Kad rotation (amule-project#795, amule-project#799/amule-project#805), GTK warning silencing (amule-project#833,
  amule-project#826/amule-project#836), and the clang-tidy worklist (amule-project#770, amule-project#772-amule-project#774).

- Translations: late-cycle wave covering French/Turkish manpages
  (amule-project#753/amule-project#754/amule-project#776), Galician (amule-project#763), Slovenian (amule-project#771), pt-BR
  (amule-project#768/amule-project#775/amule-project#812), French (amule-project#811), plus man-page tooling for
  date+version drift (amule-project#802).

- Contributors: added ngosang for UX feedback on the late-3.0
  cycle (amule-project#817/amule-project#818/amule-project#821/amule-project#828/amule-project#844) and ongoing work on the
  user-facing manual at amule-org.github.io.

- Merged PRs flat index: extended with amule-project#746-amule-project#845 + amule-project#841.
mrjimenez pushed a commit that referenced this pull request Jun 4, 2026
Adds 55+ merged PRs to the 3.0.0 changelog since the last update
(#747, 2026-05-27). Narrative additions cover:

- Packaging: expanded the top list to include the macOS per-arch .app
  bundles and the Windows NSIS installer alongside the existing
  AppImage / Flatpak / .dmg / .zip entries. New bullets for #785
  (alc/alcc/cas/wxcas everywhere + Windows amuleweb), #794 (.dmg
  amuleweb path), #789 (<OS>-<arch> artifact naming), #780 / #796
  (Windows DPI + comctl32 manifest), #784 (FHS share/amule paths).

- Bug Fixes & Stability: post-#744 fixes including EC notification
  leak (#797), big-library scaling (#736, #840 superseding #728),
  amulegui ghost entries (#810, #819, #841, #824, #830, #760),
  PartFile early hash (#762), server protocol fixes (#835, #788,
  #721, #787), crypto stream UB (#779), UAF prevention (#756),
  Kad rotation (#795, #799/#805), GTK warning silencing (#833,
  #826/#836), and the clang-tidy worklist (#770, #772-#774).

- Translations: late-cycle wave covering French/Turkish manpages
  (#753/#754/#776), Galician (#763), Slovenian (#771), pt-BR
  (#768/#775/#812), French (#811), plus man-page tooling for
  date+version drift (#802).

- Contributors: added ngosang for UX feedback on the late-3.0
  cycle (#817/#818/#821/#828/#844) and ongoing work on the
  user-facing manual at amule-org.github.io.

- Merged PRs flat index: extended with #746-#845 + #841.
got3nks added a commit to got3nks/amule that referenced this pull request Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE

The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)")
gates the per-file emit on `m_ecGen <= ec_threshold` in three
handlers. For unchanged files the server either emits a childless
alive-marker tag (legacy clients) or skips silently (partial-update
clients). Both shortcuts assume the client has previously received
the ECID with full detail. If it hasn't, the alive marker
constructs a ghost CKnownFile from an empty tag (amule-project#810 surface) and
the silent skip leaves the file invisible. The CValueMap-driven
identity suppression on subsequent emissions extends the surface to
INC_UPDATE tags that have children but no identifying fields (amule-project#819
surface).

The amulegui-side guards in amule-project#810 and amule-project#819 cover that client
specifically. Other legacy EC consumers — amuleweb, amulecmd, and
third-party clients — have no analogous guard and silently lose
newly-shared files until restart.

Fix at the source: track per-connection, per-handler ECIDs that
have been sent with full detail. Gate the alive-marker / silent-
skip shortcut on membership in that set. Files the client hasn't
seen fall through to full-detail emission unconditionally, and the
emission populates the per-connection CObjTagMap (so subsequent
INC_UPDATE diffs work) and inserts into the new sent-with-detail
set.

Three handlers, three sets — one per request path because each
handler has its own `m_lastEcGenSeen*` cadence.

Memory cost: bounded by alive-file ECID count per connection
(~360 KB at 91k files; insignificant). Never pruned — re-share
bumps m_ecGen via SetFilePath / Init so any resurrected ECID
naturally takes the full-detail branch again.

The amulegui guards from amule-project#810 and amule-project#819 stay as defensive belt-and-
suspenders for amulegui talking to unpatched / older daemons.
got3nks added a commit to got3nks/amule that referenced this pull request Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE

The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)")
gates the per-file emit on `m_ecGen <= ec_threshold` in three
handlers. For unchanged files the server either emits a childless
alive-marker tag (legacy clients) or skips silently (partial-update
clients). Both shortcuts assume the client has previously received
the ECID with full detail. If it hasn't, the alive marker
constructs a ghost CKnownFile from an empty tag (amule-project#810 surface) and
the silent skip leaves the file invisible. The CValueMap-driven
identity suppression on subsequent emissions extends the surface to
INC_UPDATE tags that have children but no identifying fields (amule-project#819
surface).

The amulegui-side guards in amule-project#810 and amule-project#819 cover that client
specifically. Other legacy EC consumers — amuleweb, amulecmd, and
third-party clients — have no analogous guard and silently lose
newly-shared files until restart.

Fix at the source: track per-connection, per-handler ECIDs that
have been sent with full detail. Gate the alive-marker / silent-
skip shortcut on membership in that set. Files the client hasn't
seen fall through to full-detail emission unconditionally, and the
emission populates the per-connection CObjTagMap (so subsequent
INC_UPDATE diffs work) and inserts into the new sent-with-detail
set.

Three handlers, three sets — one per request path because each
handler has its own `m_lastEcGenSeen*` cadence.

Memory cost: bounded by alive-file ECID count per connection
(~360 KB at 91k files; insignificant). Never pruned — re-share
bumps m_ecGen via SetFilePath / Init so any resurrected ECID
naturally takes the full-detail branch again.

The amulegui guards from amule-project#810 and amule-project#819 stay as defensive belt-and-
suspenders for amulegui talking to unpatched / older daemons.
mrjimenez pushed a commit that referenced this pull request Jun 5, 2026
…_FILES / GET_DLOAD_QUEUE

The EC skip-unchanged opt-in (93a0676 "EC skip-unchanged (4/5)")
gates the per-file emit on `m_ecGen <= ec_threshold` in three
handlers. For unchanged files the server either emits a childless
alive-marker tag (legacy clients) or skips silently (partial-update
clients). Both shortcuts assume the client has previously received
the ECID with full detail. If it hasn't, the alive marker
constructs a ghost CKnownFile from an empty tag (#810 surface) and
the silent skip leaves the file invisible. The CValueMap-driven
identity suppression on subsequent emissions extends the surface to
INC_UPDATE tags that have children but no identifying fields (#819
surface).

The amulegui-side guards in #810 and #819 cover that client
specifically. Other legacy EC consumers — amuleweb, amulecmd, and
third-party clients — have no analogous guard and silently lose
newly-shared files until restart.

Fix at the source: track per-connection, per-handler ECIDs that
have been sent with full detail. Gate the alive-marker / silent-
skip shortcut on membership in that set. Files the client hasn't
seen fall through to full-detail emission unconditionally, and the
emission populates the per-connection CObjTagMap (so subsequent
INC_UPDATE diffs work) and inserts into the new sent-with-detail
set.

Three handlers, three sets — one per request path because each
handler has its own `m_lastEcGenSeen*` cadence.

Memory cost: bounded by alive-file ECID count per connection
(~360 KB at 91k files; insignificant). Never pruned — re-share
bumps m_ecGen via SetFilePath / Init so any resurrected ECID
naturally takes the full-detail branch again.

The amulegui guards from #810 and #819 stay as defensive belt-and-
suspenders for amulegui talking to unpatched / older daemons.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

enh: don't share 0 byte entries

2 participants