fix(sanitize): keep the joiners that hold emoji and words together - #349

Summary

The invisible-character stripping in sanitize_user_content removes U+200C (ZWNJ) and U+200D (ZWJ) unconditionally (outside preserve_emoji), which mangles two legitimate uses:

• a ZWJ between two emoji parts — family, flags, profession and skin-tone sequences are single composite emoji built from ZWJ-joined symbols; stripping it splits them into unrelated emoji ("👨‍👩‍👧" → "👨👩👧"),
• a ZWNJ between letters of a joining script — in Arabic/Persian and the Indic blocks (including their presentation forms), ZWNJ is spelling, not decoration: "نمی‌روم" becomes "نمیروم".

The strip pass now keeps a ZWJ only when both neighbours are emoji parts (So symbols, VS16, skin-tone modifiers) and a ZWNJ only when both neighbours are letters of a ZWNJ-using script; everywhere else — e.g. between Latin letters, where a joiner can split a word to hide it — they are stripped as before. The separate _INVISIBLE_CHARS substitution is folded into the same single pass. With preserve_emoji, ZWJ retention is unchanged.

Testing

• tests/test_sanitize.py covers composite emoji, Persian ZWNJ spelling, and the hostile case (joiners inside Latin words still stripped).
• Full suite: 1413 passed.

🤖 Generated with Claude Code