fix: scope LC_CTYPE=C around wxFileConfig ops and country-flag lookup (#852) - #854
Conversation
…keys (amule-project#852) amule.conf accumulates duplicate `key=value` lines on sessions run under certain non-C UI locales (Turkish, reported in amule-project#852). Each duplicate appears as a contiguous block at the end of the affected section. Root cause: wxFileConfig stores entries / groups in sorted arrays and looks them up via binary search using wxString::CmpNoCase. That ultimately bottoms out at the libc case-folding family (strcasecmp / wcscasecmp), which on glibc respects LC_CTYPE. The sort order is fixed at the locale active when the file is parsed, but a session that subsequently runs wxLocale::Init (called by Localize_mule for translations) flips LC_CTYPE to the UI language via setlocale(LC_ALL, ...). For some entry-name pivots, the comparator returns different signs in C/English vs Turkish, and the binary search misnavigates the array — FindEntry returns NULL, Write interprets it as "entry doesn't exist", and AddEntry appends a duplicate to the section. Confirmed by cardpuncher: running with LC_ALL=C suppresses the duplicates entirely, narrowing the bug to a case-folding category of the locale. LC_ALL=C globally is too aggressive — it would also disable translations, date / number formatting, and locale-aware filename sort in the GUI list controls. Scope the lock to wxFileConfig operations only: - New CamuleFileConfig (header-only) subclasses wxFileConfig and overrides every Read / Write / Has / Delete / Rename / SetPath / Flush method to wrap each call in a CCtypeAsciiScope RAII guard that sets LC_CTYPE="C" on entry and restores the previous value on destruction. - amuleAppCommon::Initialize now creates a CamuleFileConfig and wraps the constructor itself in the same RAII guard so the initial file parse establishes its in-memory sort order under LC_CTYPE="C" too — otherwise post-parse lookups would still desync from a parse done in a different locale. Other LC_* categories (LC_MESSAGES, LC_TIME, LC_NUMERIC, LC_COLLATE) keep following the user's UI language, so translations / date formatting / filename sort in list controls stay locale-aware.
The helper now has a second use case (locale-safe lowercasing of ISO country codes in the libmaxminddb path), so the file it lived in is no longer the right home. Move CCtypeAsciiScope verbatim into src/CCtypeAsciiScope.h; have CamuleFileConfig.h include it. No behaviour change.
CMaxMindDBDatabase::GetCountryISOCode returns a lowercased copy of the ISO 3166-1 alpha-2 code from the .mmdb. The result feeds CamuleArtProvider's "flag_<cc>" embedded-bitmap lookup, so the lowercase has to be ASCII. Under a Turkish locale (tr_TR), wxString::Lower() folds 'I' to U+0131 (dotless i) rather than 'i'. "IT" becomes "ıt" and the flag bitmap is not found. This is the same Turkish-I bug class as amule-project#852 but a different code path. SVN r10697 fixed the original instance in the libGeoIP path in 2011; the fix was lost when GeoIP was replaced by libmaxminddb in PR amule-project#502. Wrap the Lower() call with CCtypeAsciiScope (introduced in the previous commit) so the case fold runs under LC_CTYPE=C regardless of the user's UI language. Forum thread: http://forum.amule.org/index.php?topic=19398.0
|
Folded in a second fix while we have the Turkish-locale shape in mind. cardpuncher pointed at http://forum.amule.org/index.php?topic=19398.0 — country flags also fail in tr_TR because |
|
I downloaded https://github.com/got3nks/amule/archive/refs/heads/fix/wxconfig-ctype-c-scope.tar.gz extracted, compiled and installed it after removing the previous version. Also nuked The duplicated keys are gone :) |
|
Could you test the follow-up fix for the GeoIP flags too? @cardpuncher |
|
I'm trying. I got your version, have libmaxminddb-devel installed and I copied a GeoLite2-Country.mmdb in ~/.aMule/ but how to build aMule with the country flags enabled? When I add -DBUILD_ENABLE_IP2COUNTRY=YES I get a warning: Also, just for your information, when compiling everything was clean except these warnings: |
|
The flag is The For the Turkish-locale check specifically, please run amule as |
|
Thanks. I can now confirm that the Italian flag renders correctly. And I should learn to read better, -DENABLE was indicated in INSTALL.md :) |
|
Just tested with this build after cycling from Turkish to French, no duplicated keys either in aMule log. Thanks again for the lightning fast fix. |
|
@mrjimenez this can be merged. |
…ule-project#912) Extends existing categories (preferring extensions over new lines): - Performance/Upload: amule-project#898 SlotAllocation default raised. - Networking & Discovery: wire-parser hardening list extended with amule-project#879/amule-project#882/amule-project#890/amule-project#886; new amuleweb security hardening bullet consolidating ngosang's amule-project#869-amule-project#874 triage (all landed in amule-project#875); amulegui list extended with amule-project#857; shared-folder watcher extended with amule-project#858. - Packaging: Windows installer i18n line extended with amule-project#899. - Internals & Refactoring: new docs-polish + code-quality bullets covering amule-project#851/amule-project#855/amule-project#862/amule-project#888/amule-project#900/amule-project#866/amule-project#867/amule-project#895 and amule-project#909/amule-project#910/amule-project#912. - Translations: new pre-release final-wave bullet covering amule-project#847/amule-project#856/ amule-project#891/amule-project#908/amule-project#860/amule-project#904/amule-project#859/amule-project#863/amule-project#861/amule-project#880/amule-project#911/amule-project#901/amule-project#902/amule-project#889/amule-project#868/amule-project#853. - Bug Fixes & Stability: amule-project#850/amule-project#854/amule-project#878/amule-project#906. - CI: ccache wiring (amule-project#892, amule-project#903) + CodeQL binutils-dev (amule-project#907). Contributors footer gains mifritscher and nguyenhoangminhhieu2004-gif (both first-time contributors). PR index extended through amule-project#912.
Extends existing categories (preferring extensions over new lines): - Performance/Upload: #898 SlotAllocation default raised. - Networking & Discovery: wire-parser hardening list extended with #879/#882/#890/#886; new amuleweb security hardening bullet consolidating ngosang's #869-#874 triage (all landed in #875); amulegui list extended with #857; shared-folder watcher extended with #858. - Packaging: Windows installer i18n line extended with #899. - Internals & Refactoring: new docs-polish + code-quality bullets covering #851/#855/#862/#888/#900/#866/#867/#895 and #909/#910/#912. - Translations: new pre-release final-wave bullet covering #847/#856/ #891/#908/#860/#904/#859/#863/#861/#880/#911/#901/#902/#889/#868/#853. - Bug Fixes & Stability: #850/#854/#878/#906. - CI: ccache wiring (#892, #903) + CodeQL binutils-dev (#907). Contributors footer gains mifritscher and nguyenhoangminhhieu2004-gif (both first-time contributors). PR index extended through #912.
…amule-project#854) * feat(gui): amuleGUI remote-to-local path mappings (amule-project#843) FileLaunch::Open()/Reveal() silently disable themselves whenever the daemon-reported path doesn't exist verbatim on amuleGUI's own machine, which is always true for a genuinely remote daemon unless the exact same absolute path happens to be independently mounted. Lets the user configure a table of remote->local path-prefix mappings (e.g. the daemon's /downloads/incoming reachable here as D:\Downloads\aMule\incoming via a Samba mount) so those actions work against a remote daemon whose filesystem is otherwise reachable. Storage is genuinely GUI-local: CPreferences::PathMapping / Load|SavePathMappings() read and write wxConfigBase::Get() directly (which already resolves to remote.conf under CLIENT_GUI), never through LoadAllItems()/SaveAllItems()'s Cfg_Base walk and never added to CPreferencesRem's m_exchange_send_selected_prefs / m_exchange_recv_selected_prefs -- so this never round-trips over EC. Two existing "list of items" precedents looked reusable but weren't: Categories and the shared-dirs editor both look locally persisted but are actually daemon-owned, silently overwritten by the next EC pull. Took SaveCats()'s per-row wxConfigBase group shape without its EC-backed data source. Applied in FileLaunch::ResolvePath(), the single choke point every Open/Reveal/availability check already goes through (six call sites across DownloadListCtrl.cpp and SharedFilesCtrl.cpp, none touched). ApplyPathMapping() does plain string-prefix substitution on the raw daemon path before it becomes a CPath -- CPath has no notion of a second machine's separator convention, so the remote side is compared and rewritten as a string, never parsed as this host's path syntax. First matching prefix in the user's list order wins (no implicit longest-prefix-match). New "Path Mappings" Preferences page, CLIENT_GUI-only, modelled on the existing shared-dirs list editor's widget shape (wxListCtrl + text entry + Add/Remove) but without its EC round-trip/dirty-flag/session machinery, which exists only to survive a background daemon refresh that cannot happen to a purely local list. Known gap: developed and tested on macOS. The substitution itself is separator-agnostic pure string logic, and mingw-w64 CI confirms the Windows build compiles, but an actual Samba-mounted drive-letter/UNC mapping has not been exercised end-to-end this round. Verified: amule (CLIENT_GUI off) and amulegui (CLIENT_GUI on) build clean from a fresh CMake configure; pinned clang-format v18 clean; Tier-1 clang-tidy clean via the local CI replica against upstream/master (caught and fixed one real hit: an unconditional CPath copy in ResolvePath() that's wasted work on the non-CLIENT_GUI build, where the mapping branch never runs). * fix(gui): address review findings on amule-project#843 path mappings, rebase onto master got3nks's review on amule-project#854 (github.com/amule-org/pull/854): 1. A trailing separator on the remote prefix silently corrupted every path it mapped: the substitution is localPrefix + remainder, and CPath's constructor does not strip trailing separators, so remote "/downloads/" + local "/Volumes/dl" produced "/Volumes/dlincoming/f.avi" instead of "/Volumes/dl/incoming/f.avi". Browse never supplies a trailing separator (wxDirDialog::GetPath() doesn't return one) but a user describing a remote *directory* naturally types one, so this was the likely path through the UI, not a corner case. Fixed by stripping trailing separators from both prefixes wherever they're accepted: OnPathMappingAdd() (entry) and LoadPathMappings() (an existing config saved before this fix, or hand-edited). 2. ApplyPathMapping()'s prefix match had no boundary check: a mapping for "/mnt/data" also matched "/mnt/data-old/f.avi". Fixed by requiring the character after the prefix to be a separator (either convention, since the daemon's OS isn't known here) or the whole string to match. 3. The explanatory paragraph above the mapping list didn't use the panel width: muuli_wdr.cpp's one-shot Wrap(380) baked fixed line breaks at construction while the sizer's Expand() only stretched the control, leaving an empty right margin, and never re-flowed on resize (the dialog has wxRESIZE_BORDER) or DPI. Fixed by dropping the Wrap() from muuli_wdr.cpp (layout construction only, per that file's convention) and re-wrapping from the control's own real, DPI-scaled client width in PrefsUnifiedDlg -- once in PopulatePathMappingList() and again on every resize via a runtime Bind(wxEVT_SIZE) (size events don't propagate through the static event table the way command events do, so this can't be a wxDECLARE_EVENT_TABLE() row). Minor: OnPathMappingAdd() also now tells the user why nothing happened on empty/duplicate input instead of silently no-opping. Also rebased onto current master to pick up amule-project#859, which fixed the same "cell text is not a round-trippable CPath" bug (macOS NFD-normalisation mismatch) in the shared-dirs editor this one was modelled on, and extracted SetListRowPath()/GetListRowPath() for exactly this reuse. Path-mapping's local-prefix column now goes through them instead of GetPrintable()/CPath(text) round-tripping by hand. Verified: rebuilt amule, amuled and amulegui locally, all clean. po/ regenerated -- msgid delta +13/-0 (11 carried over from the original amule-project#843 strings the rebase's po/ conflict resolution reset to upstream, plus 2 new validation-message strings from this fix). * fix(gui): re-flow the path-mapping hint from the page, not from itself The re-wrap added for the review's layout point read the paragraph's own width and rewrote the paragraph from inside that paragraph's size handler. wxStaticText::SetLabel() resizes the control to fit its new label, so the handler fed itself: restoring the unwrapped text to re-flow made the control briefly as wide as the whole sentence, that width came back as another size event, and the two alternated until the stack was gone (EXC_BAD_ACCESS, "excessive recursion", inside SetLabel under NSView setFrameSize). Bound on the page instead. A page's width is set by the dialog and is unmoved by anything its children do, so the input to the wrap no longer depends on what the wrap changes and the loop is gone structurally rather than by hoping the widths converge. Re-wrapping is additionally skipped when the width has not moved, so a height-only relayout costs nothing, and page->Layout() runs afterwards so the list follows the paragraph's new height. Two things the earlier version could not do, now fixed with it: Wrap() only ever inserts breaks -- it re-reads the current label and treats newlines already in it as hard -- so wrapping in place could narrow the text but never rejoin it. Widening the dialog left the paragraph at its old narrow width, which is the empty right margin the review was about, just reached by resizing instead of baked in. The unwrapped text is kept and restored before each wrap. muuli_wdr.cpp wraps once again at construction, which the fix had dropped. An unwrapped wxStaticText reports its whole single line as its best width and the sizer turns that into the page's minimum, so the dialog opened as wide as the sentence. It is bounded there and re-flowed from here; the bound is parent->FromDIP(380) rather than a raw pixel count, which was the part that read cramped on a HiDPI display. Also adds the include for StripSeparators, which Preferences.cpp had been getting transitively. Tested on macOS: the page opens at a sane width, the paragraph fills it, and it re-flows both wider and narrower under a continuous drag. * fix(gui): make path mapping work in both directions across OS conventions Two gaps the trailing-separator trim left, both about the mapping's two halves coming from machines that need not agree on a separator. The trim used StripSeparators(), which consults *this* host's separator set. On a POSIX build that set has no backslash, so a Windows daemon's "D:\dl\" kept its trailing separator and the join ran the halves together -- exactly the corruption the trim exists to prevent, alive in the mirror direction (a Linux or macOS amulegui against a Windows daemon). The remote prefix is now trimmed by CPreferences::TrimRemotePrefix(), which accepts either convention because the daemon's OS is not knowable here -- the same reason the prefix boundary test already accepts either character. And the remainder spliced onto the local prefix keeps the daemon's separators, so a POSIX daemon feeds '/' into a path about to be handed to Win32. Most of Win32 takes that, but "explorer /select," -- which is what Reveal() runs, and the reason this feature exists -- is the fussy one. Normalised on Windows only: '/' cannot occur in a Windows filename, whereas a backslash is an ordinary character in a POSIX one, so the mirror rewrite would corrupt names instead of fixing separators. Verified on Windows that Win32 resolves mixed separators (Test-Path on "C:\dir/sub/f.txt" is true), so availability and Open were already fine there; this is for Reveal and for the POSIX-client-Windows-daemon direction. --------- Co-authored-by: got3nks <[email protected]>
Closes #852.
Symptom
amule.confaccumulates duplicatekey=valuelines on sessions run under certain non-C UI locales (Turkish, in cardpuncher's report). Each duplicate appears as a contiguous block at the end of the affected section ([eMule],[ExternalConnect],[Razor_Preferences]).Root cause
wxFileConfigstores entries and groups in sorted arrays and looks them up via binary search usingwxString::CmpNoCase. That ultimately bottoms out at the libc case-folding family (strcasecmp/wcscasecmp), which on glibc respectsLC_CTYPE.The sort order is fixed at the locale active when the file is parsed, but a session that subsequently calls
wxLocale::Init()(whichLocalize_mule()does for translations) flipsLC_CTYPEto the UI language viasetlocale(LC_ALL, ...). For some pivots, the comparator returns different signs in C/English vs Turkish — the binary search misnavigates,FindEntryreturnsNULL,Writeinterprets it as "entry doesn't exist", andAddEntryappends a duplicate to the section.@cardpuncher confirmed by running with
LC_ALL=C— duplicates vanish, narrowing the bug to the locale's case-folding category.Fix
LC_ALL=Cglobally is too aggressive (would disable translations, date/number formatting, and locale-aware filename sort in list controls). Scope the lock to wxFileConfig operations only:CamuleFileConfig(header-only) subclasseswxFileConfigand overrides every Read / Write / Has / Delete / Rename / SetPath / Flush method to wrap each call in aCCtypeAsciiScopeRAII guard that setsLC_CTYPE="C"on entry and restores the previous value on destruction.amuleAppCommon::Initializenow creates aCamuleFileConfigand wraps the constructor itself in the same RAII guard, so the initial parse establishes its in-memory sort order underLC_CTYPE="C"too — otherwise post-parse lookups would still desync from a parse done in a different locale.Other
LC_*categories (LC_MESSAGES,LC_TIME,LC_NUMERIC,LC_COLLATE) keep following the user's UI language, so translations / date formatting / filename sort in list controls stay locale-aware.Verified
cmake --build build-macos) clean.LC_ALL=Cthis time):appears more than oncelines.