Thank you for your contribution! Before we can merge, please sign the Contributor License Agreement.

To sign, comment on this pull request with the statement below. You only need to sign once per GitHub account.

I have read the CLA Document and I hereby sign the CLA

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

Thanks for the careful fix and the thorough test file — really appreciate the parametrized regex coverage plus the end-to-end cases.

I ran the tests locally on the PR branch:

• pytest test_fts_tokenizer.py → 45/45 pass
• pre-existing test_usability_fixes.py::TestFTS5HyphenQuoting → 4/4 still pass
• broader pytest -k "fts or tokeniz or bm25 or search" → 218/218 pass, no adjacent regressions
• mutation check (revert the regex to old, re-run new tests) → 25 of 45 fail — confirms the tests genuinely pin the new behavior

Required before merge

• Sign the CLA — the bot comment up-thread has the exact text to copy.

• uv run ruff format — ruff format --check currently fails on three spots, all auto-fixable in one command:

  1. fts_tokenizer.py:18 — single-quoted regex string → ruff prefers double-quoted (r"...").
  2. test_fts_tokenizer.py — class definitions need two blank lines before them (PEP 8) at lines 24, 46, 95.
  3. test_fts_tokenizer.py:90 — trailing whitespace on the 'say* """hello"""' assertion line.

  Note: ruff check already passes — it's only ruff format --check that flags these. Both are gating in CI.

Suggestion (non-blocking)

Could you add a ### Fixed line under [Unreleased] in the top-level CHANGELOG.md? Something like:

• mem_search BM25 leg now handles queries containing . / \ < > ~ (and the rest of the unicode61 separator set) — previously silently fell back to dense-only with a WARNING log. (fix: extend FTS5 special-char regex to cover dot, slash, backslash, angle brackets, tilde #715, fixes BM25 FTS5 query sanitization misses . / \ < > (silent degradation) #697)

Out of scope (just noting for future, not for this PR)

• The kiwipiepy backend at fts_tokenizer.py:101 drops tokens matching the regex rather than quoting them. That pre-dates this PR, but the silent-drop surface widens as the regex grows — worth a follow-up.
• The if safe: branch at fts_tokenizer.py:123 is dead (text.split() never yields empty, .replace only grows). Pre-existing.

Approve

Once the format pass + CLA land, happy to approve and merge. Thanks again for picking this up!