What

Add support for Jina v3 reranker models (jinaai/jina-reranker-v3-mlx) to omlx's reranker module.

Why

Issue #326 requested Jina reranker support. The Jina v3 reranker uses <|score_token|> logits for scoring instead of the yes/no token approach used by Qwen3-Reranker, and requires a separate implementation path.

Changes

• Add JinaForRanking to CAUSAL_LM_RERANKER_ARCHITECTURES in model_discovery.py
• Add MLXRerankerModel._load_jina_reranker() for loading Jina models via mlx-lm
• Add MLXRerankerModel._rerank_jina() for reranking using <|score_token|> logits
• Dispatch to the Jina path when architecture is JinaForRanking
• Fix token masking and BOS/EOS handling in the Jina rerank implementation
• Fix token ID resolution to handle different tokenizer formats (str vs Token objects in added_tokens_decoder)

Implementation Notes

• Jina v3 reranker is based on Qwen3 architecture and uses <|score_token|> and <|rerank_token|> special tokens
• The scoring extracts logits at the <|score_token|> position at the last token
• The implementation follows the same pattern as the existing _rerank_causal_lm() but with Jina-specific token handling
• Falls back to existing reranker implementations for other architectures (no breaking changes)

Validation (Local testing PASSED ✅)

This implementation has been validated with a real jinaai/jina-reranker-v3-mlx model:

• ✅ Syntax checked with omlx's bundled Python environment
• ✅ End-to-end tested with actual model (downloaded via Hugging Face Hub)
• ✅ Functional verification: Model loads successfully, token IDs resolved correctly (<|score_token|> = 151669, <|rerank_token|> = 151671)
• ✅ Reranking works: Produces sensible scores with correct sorting
• ✅ Code patterns: Follows existing patterns in reranker.py

Testing Results

The implementation was tested locally with:

• Model: jinaai/jina-reranker-v3-mlx
• Query: "What is artificial intelligence?"
• Documents: 3 sample texts about AI, ML, and Python
• Result: Successful loading and reranking with correct token resolution and score sorting

Notes

• Token ID resolution now handles multiple tokenizer formats (string tokens, Token objects, dict entries)
• Fallback mechanisms: added_tokens_decoder → convert_tokens_to_ids → get_added_vocab

Closes: #326