Skip to content

amuled depends on the process locale for UTF-8 handling; mangles non-ASCII filenames under the default C locale #203

Description

@ngosang

Summary

When amuled is started without a UTF-8 locale in the environment (no LANG / LC_ALL,
i.e. the default C/POSIX locale), non-ASCII text such as filenames and server messages is
mangled. For example a file named La.mujer.danesa.1x02.Un.niño.destrozado is shown as
La.mujer.danesa.1x02.Un.ni\361o (\361 = byte 0xF1, ñ in Latin-1).

This is common in headless deployments: Docker containers, systemd services, cron jobs and
minimal servers frequently leave LANG/LC_ALL unset, so amuled inherits the C locale.
Setting LANG=C.UTF-8 in the environment fixes it, which confirms the root cause is a locale
dependency inside aMule rather than the data itself.

This was originally reported downstream in the Docker image:
ngosang/docker-amule#98

Root cause

amuled relies on the ambient locale for wxStringchar* conversions, but never
initialises it:

  1. The daemon never calls setlocale(LC_ALL, ""). Only the external connectors do, in
    src/ExternalConnector.cpp (setlocale(LC_ALL, "")), and the webserver via
    CaMuleExternalConnector::SetLocale(). CamuleDaemonApp::Initialize()
    (src/amuled.cpp) reads wxLocale::GetSystemEncodingName() only to set
    wxConvFileName; it does not touch LC_CTYPE. So when launched without LANG/LC_ALL,
    amuled stays in the C locale for its entire lifetime.

  2. Most conversions go through wxConvLibc, which is LC_CTYPE-dependent.
    unicode2char() / char2unicode() in src/libs/common/StringFunctions.{h,cpp} use
    wxConvLibc. In the C locale wxConvLibc has no mapping for bytes > 127, so non-ASCII
    codepoints are dropped, replaced with ?, or escaped. There are ~95 unicode2char() call
    sites across the daemon (core and network code), so this affects far more than logging.

  3. The recent fix amuled -o corrupts non-ASCII log lines on stdout (uses C-locale wxConvLibc, never calls setlocale); on-disk logfile is fine #40 / fix(logger): write stdout/stderr via utf8_str instead of wxConvLibc (#40) #42 switched the stdout/stderr log sink to utf8_str()
    (src/Logger.cpp), which is locale-independent. That fixes the log output specifically,
    but every other path that still uses wxConvLibc remains locale-dependent, so the
    underlying problem is only partially addressed.

Steps to reproduce

  1. Build/run amuled in an environment with the default C locale (e.g. a minimal Docker
    container with LANG/LC_ALL unset). Verify with locale (shows LC_CTYPE="C").
  2. Download / share a file whose name contains non-ASCII characters (e.g. niño).
  3. Observe the name in the daemon output / amulecmd / the web UI: the non-ASCII characters
    appear mangled (ni\361o, ?, or similar).
  4. Restart amuled with LANG=C.UTF-8 (or any *.UTF-8 locale) in the environment and
    repeat — the name is now correct. This is the difference that pins the cause to the locale.

Environment: aMule 3.0.0 (daemon / amuled), Linux, default C locale (no LANG/LC_ALL).
On-disk and source data are UTF-8.

Suggested fix

Make amuled independent of whatever locale the environment happens to provide, e.g.:

The aim is that a default C locale should not corrupt non-ASCII filenames or messages.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions