You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When amuled is started without a UTF-8 locale in the environment (no LANG / LC_ALL,
i.e. the default C/POSIX locale), non-ASCII text such as filenames and server messages is
mangled. For example a file named La.mujer.danesa.1x02.Un.niño.destrozado is shown as La.mujer.danesa.1x02.Un.ni\361o (\361 = byte 0xF1, ñ in Latin-1).
This is common in headless deployments: Docker containers, systemd services, cron jobs and
minimal servers frequently leave LANG/LC_ALL unset, so amuled inherits the C locale.
Setting LANG=C.UTF-8 in the environment fixes it, which confirms the root cause is a locale
dependency inside aMule rather than the data itself.
amuled relies on the ambient locale for wxString ⇄ char* conversions, but never
initialises it:
The daemon never calls setlocale(LC_ALL, ""). Only the external connectors do, in src/ExternalConnector.cpp (setlocale(LC_ALL, "")), and the webserver via CaMuleExternalConnector::SetLocale(). CamuleDaemonApp::Initialize()
(src/amuled.cpp) reads wxLocale::GetSystemEncodingName() only to set wxConvFileName; it does not touch LC_CTYPE. So when launched without LANG/LC_ALL, amuled stays in the C locale for its entire lifetime.
Most conversions go through wxConvLibc, which is LC_CTYPE-dependent. unicode2char() / char2unicode() in src/libs/common/StringFunctions.{h,cpp} use wxConvLibc. In the C locale wxConvLibc has no mapping for bytes > 127, so non-ASCII
codepoints are dropped, replaced with ?, or escaped. There are ~95 unicode2char() call
sites across the daemon (core and network code), so this affects far more than logging.
Build/run amuled in an environment with the default C locale (e.g. a minimal Docker
container with LANG/LC_ALL unset). Verify with locale (shows LC_CTYPE="C").
Download / share a file whose name contains non-ASCII characters (e.g. niño).
Observe the name in the daemon output / amulecmd / the web UI: the non-ASCII characters
appear mangled (ni\361o, ?, or similar).
Restart amuled with LANG=C.UTF-8 (or any *.UTF-8 locale) in the environment and
repeat — the name is now correct. This is the difference that pins the cause to the locale.
Environment: aMule 3.0.0 (daemon / amuled), Linux, default C locale (no LANG/LC_ALL).
On-disk and source data are UTF-8.
Suggested fix
Make amuled independent of whatever locale the environment happens to provide, e.g.:
Call setlocale(LC_ALL, "") (or specifically LC_CTYPE) early in the daemon startup, the
same way the external connectors already do, so it picks up the environment's UTF-8 locale
when one is configured; and/or
Summary
When
amuledis started without a UTF-8 locale in the environment (noLANG/LC_ALL,i.e. the default
C/POSIXlocale), non-ASCII text such as filenames and server messages ismangled. For example a file named
La.mujer.danesa.1x02.Un.niño.destrozadois shown asLa.mujer.danesa.1x02.Un.ni\361o(\361= byte0xF1,ñin Latin-1).This is common in headless deployments: Docker containers, systemd services, cron jobs and
minimal servers frequently leave
LANG/LC_ALLunset, soamuledinherits theClocale.Setting
LANG=C.UTF-8in the environment fixes it, which confirms the root cause is a localedependency inside aMule rather than the data itself.
This was originally reported downstream in the Docker image:
ngosang/docker-amule#98
Root cause
amuledrelies on the ambient locale forwxString⇄char*conversions, but neverinitialises it:
The daemon never calls
setlocale(LC_ALL, ""). Only the external connectors do, insrc/ExternalConnector.cpp(setlocale(LC_ALL, "")), and the webserver viaCaMuleExternalConnector::SetLocale().CamuleDaemonApp::Initialize()(
src/amuled.cpp) readswxLocale::GetSystemEncodingName()only to setwxConvFileName; it does not touchLC_CTYPE. So when launched withoutLANG/LC_ALL,amuledstays in theClocale for its entire lifetime.Most conversions go through
wxConvLibc, which isLC_CTYPE-dependent.unicode2char()/char2unicode()insrc/libs/common/StringFunctions.{h,cpp}usewxConvLibc. In theClocalewxConvLibchas no mapping for bytes > 127, so non-ASCIIcodepoints are dropped, replaced with
?, or escaped. There are ~95unicode2char()callsites across the daemon (core and network code), so this affects far more than logging.
The recent fix amuled -o corrupts non-ASCII log lines on stdout (uses C-locale wxConvLibc, never calls setlocale); on-disk logfile is fine #40 / fix(logger): write stdout/stderr via utf8_str instead of wxConvLibc (#40) #42 switched the stdout/stderr log sink to
utf8_str()(
src/Logger.cpp), which is locale-independent. That fixes the log output specifically,but every other path that still uses
wxConvLibcremains locale-dependent, so theunderlying problem is only partially addressed.
Steps to reproduce
amuledin an environment with the defaultClocale (e.g. a minimal Dockercontainer with
LANG/LC_ALLunset). Verify withlocale(showsLC_CTYPE="C").niño).amulecmd/ the web UI: the non-ASCII charactersappear mangled (
ni\361o,?, or similar).amuledwithLANG=C.UTF-8(or any*.UTF-8locale) in the environment andrepeat — the name is now correct. This is the difference that pins the cause to the locale.
Environment: aMule 3.0.0 (daemon /
amuled), Linux, defaultClocale (noLANG/LC_ALL).On-disk and source data are UTF-8.
Suggested fix
Make
amuledindependent of whatever locale the environment happens to provide, e.g.:setlocale(LC_ALL, "")(or specificallyLC_CTYPE) early in the daemon startup, thesame way the external connectors already do, so it picks up the environment's UTF-8 locale
when one is configured; and/or
wxConvUTF8/utf8_str()) instead ofwxConvLibconthe paths that currently depend on the locale, so correctness no longer hinges on
LC_CTYPEat all (as was already done for the log sink in amuled -o corrupts non-ASCII log lines on stdout (uses C-locale wxConvLibc, never calls setlocale); on-disk logfile is fine #40 / fix(logger): write stdout/stderr via utf8_str instead of wxConvLibc (#40) #42).The aim is that a default
Clocale should not corrupt non-ASCII filenames or messages.