Description
The DMS rspamd setup script (/usr/local/bin/setup.d/security/rspamd.sh line 128) sets expand_keys = true in /etc/rspamd/local.d/redis.conf:
cat >"${RSPAMD_LOCAL_D}/redis.conf" << "EOF"
# documentation: https://rspamd.com/doc/configuration/redis.html
servers = "127.0.0.1:6379";
expand_keys = true;
EOF
This causes rspamd's lutil.template() function to be applied to all EVALSHA key arguments, including binary msgpack data used by the Bayes classifier. The template function uses $ as a variable marker, so any 0x24 byte in binary data gets stripped — corrupting msgpack serialization and causing "Missing bytes in input" errors in Bayes classification and learning.
Impact
- Bayes classification fails with "Missing bytes in input" for users whose per-user token keys happen to be exactly 36 bytes long (because 36 = 0x24 =
$ in ASCII, which is the msgpack str8 length byte)
- Even when classification appears to succeed, silent data corruption degrades Bayes accuracy (individual ASCII bytes returned as numbers instead of proper token keys)
- Affects any DMS installation using
per_user Bayes classification with expand_keys = true (the default)
Root cause details
See rspamd issue: rspamd/rspamd#5904 (comment)
The core problem is an interaction between two things:
- rspamd passes binary msgpack data as KEYS (not ARGV) in EVALSHA calls
expand_keys = true causes lutil.template() to process all EVALSHA KEYS, including binary data
The rspamd side should be fixed to not pass binary data as KEYS, but in the meantime, expand_keys = true is unsafe for any rspamd module that passes binary data through EVALSHA.
Suggested fix
Either:
- Remove
expand_keys = true from the generated redis.conf (revert to rspamd's default of false), or
- Only set it when actually needed — no default DMS module appears to use template variables (
${principal_recipient_domain} etc.) in Redis key names
Workaround
Users can override by placing a redis.conf in their rspamd local.d config directory with expand_keys = false, and copying it via user-patches.sh:
# In user-patches.sh
if [ -d /tmp/docker-mailserver/rspamd/local.d ]; then
cp /tmp/docker-mailserver/rspamd/local.d/* /etc/rspamd/local.d/
fi
# /path/to/config/rspamd/local.d/redis.conf
servers = "127.0.0.1:6379";
expand_keys = false;
Environment
- docker-mailserver v15
- rspamd 3.12.1 (bundled) and 3.14.3 (upgraded via apt) — both affected
- Per-user Bayes classification enabled
Description
The DMS rspamd setup script (
/usr/local/bin/setup.d/security/rspamd.shline 128) setsexpand_keys = truein/etc/rspamd/local.d/redis.conf:This causes rspamd's
lutil.template()function to be applied to all EVALSHA key arguments, including binary msgpack data used by the Bayes classifier. The template function uses$as a variable marker, so any0x24byte in binary data gets stripped — corrupting msgpack serialization and causing "Missing bytes in input" errors in Bayes classification and learning.Impact
$in ASCII, which is the msgpack str8 length byte)per_userBayes classification withexpand_keys = true(the default)Root cause details
See rspamd issue: rspamd/rspamd#5904 (comment)
The core problem is an interaction between two things:
expand_keys = truecauseslutil.template()to process all EVALSHA KEYS, including binary dataThe rspamd side should be fixed to not pass binary data as KEYS, but in the meantime,
expand_keys = trueis unsafe for any rspamd module that passes binary data through EVALSHA.Suggested fix
Either:
expand_keys = truefrom the generated redis.conf (revert to rspamd's default offalse), or${principal_recipient_domain}etc.) in Redis key namesWorkaround
Users can override by placing a
redis.confin their rspamdlocal.dconfig directory withexpand_keys = false, and copying it viauser-patches.sh:Environment