Jingbiao Mei

Jingbiao Mei

PhD student at University of Cambridge

Publications

You can also find my articles on my Google Scholar profile.

Conference Papers

Retrieval-Augmented Defense: Adaptive and Controllable Jailbreak Prevention for Large Language Models

Published in ACL 2026 Main

We introduce RAD, a retrieval-augmented defense mechanism for preventing jailbreak attacks on large language models. Our approach adaptively retrieves relevant safety guidelines to provide controllable and effective jailbreak prevention.

Recommended citation: G. Yang, J. Chen, J. Mei, W. Lin, B. Byrne. "Retrieval-Augmented Defense: Adaptive and Controllable Jailbreak Prevention for Large Language Models." ACL 2026 Main.
Download Paper

ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection

Published in ICLR 2026

We propose ExPO-HM, a novel explain-then-detect framework for hateful meme detection. By first generating explanations of why a meme might be hateful and then using these explanations to guide detection, our model achieves improved performance and interpretability.

Recommended citation: J. Mei, M. Sun, J. Chen, P. Qin, Y. Li, D. Chen, B. Byrne. "ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection." ICLR 2026.

On Extending Direct Preference Optimization to Accommodate Ties

Published in NeurIPS 2025

We extend Direct Preference Optimization (DPO) to handle tied preferences, where annotators cannot distinguish between two responses. Our method provides a principled framework for incorporating ties into preference-based alignment of language models.

Recommended citation: J. Chen, G. Yang, W. Lin, J. Mei, B. Byrne. "On Extending Direct Preference Optimization to Accommodate Ties." NeurIPS 2025.

Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection

Published in EMNLP 2025 Main (Oral)

We propose a robust adaptation approach for large multimodal models in retrieval-augmented hateful meme detection. Our method effectively leverages retrieved examples to improve detection performance while maintaining robustness against noisy retrievals.

Recommended citation: J. Mei, J. Chen, G. Yang, W. Lin, B. Byrne. "Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection." EMNLP 2025 Main (Oral).

Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning

Published in ACL 2024 Main

We propose RGCL, a retrieval-guided contrastive learning framework for hateful meme detection. By retrieving similar memes and using them as contrastive examples, our model learns more discriminative representations for distinguishing hateful from benign memes.

Recommended citation: J. Mei, J. Chen, W. Lin, B. Byrne, M. Tomalin. "Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning." ACL 2024 Main.
Download Paper

PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-Modal Retrievers

Published in ACL 2024 Main

We introduce PreFLMR, a large-scale pre-trained fine-grained late-interaction multi-modal retriever. By scaling up both the model and pre-training data, PreFLMR achieves state-of-the-art performance on a wide range of multi-modal retrieval benchmarks.

Recommended citation: W. Lin*, J. Mei*, J. Chen*, B. Byrne. "PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-Modal Retrievers." ACL 2024 Main.
Download Paper

Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata

Published in NAACL 2024 Main

We introduce Control-DAG, a constrained decoding method for non-autoregressive Directed Acyclic T5 using weighted finite state automata. This enables efficient constrained generation while maintaining the speed advantages of non-autoregressive models.

Recommended citation: J. Chen, W. Lin, J. Mei, B. Byrne. "Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata." NAACL 2024 Main.
Download Paper

Fine-Grained Late-Interaction Multi-Modal Retrieval for Retrieval Augmented Visual Question Answering

Published in NeurIPS 2023

We propose FLMR, a fine-grained late-interaction multi-modal retrieval model for retrieval-augmented visual question answering. FLMR extends the ColBERT architecture to handle multi-modal queries and documents, enabling effective retrieval for knowledge-intensive VQA tasks.

Recommended citation: W. Lin, J. Chen, J. Mei, A. Coca, B. Byrne. "Fine-Grained Late-Interaction Multi-Modal Retrieval for Retrieval Augmented Visual Question Answering." NeurIPS 2023.
Download Paper

BayesFT: Bayesian Optimization for Fault Tolerant Neural Network Architecture

Published in DAC 2021 (Design Automation Conference)

We propose BayesFT, a Bayesian optimization framework for designing fault-tolerant neural network architectures. Our approach efficiently searches the architecture space to find models that maintain performance under hardware faults, achieving state-of-the-art results on fault-tolerant neural network design.

Recommended citation: N. Ye*, J. Mei*, Z. Fang, Y. Zhang, Z. Zhang, H. Wu, X. Liang. "BayesFT: Bayesian Optimization for Fault Tolerant Neural Network Architecture." DAC 2021.

Preprints

Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)

Published in arXiv preprint

This report describes the Multimodal Document Retrieval Challenge (Track 1 of the MIR Challenge) at the first EReL@MIR workshop, co-located with The Web Conference 2025: challenge design, datasets, and evaluation protocol across closed-set document page retrieval (MMDocIR) and open-domain image-based retrieval (M2KR), final standings, and an analysis of the three winning systems. The challenge drew 22 teams, 455 entrants, and 586 submissions; all three winning systems build on decoder-based Multimodal-LLM embedders from the Qwen2-VL family rather than CLIP-style encoders.

Recommended citation: J. Mei. "Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)." arXiv preprint arXiv:2606.04240.
Download Paper

BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

Published in arXiv preprint

A retrieval-augmented generation framework that conditions the language model on each retrieved document separately rather than concatenating them into one context, then combines the outputs as a Bayesian ensemble whose weights are document posteriors updated during generation. This avoids the lost-in-the-middle effect of long contexts, yields document-level attribution, and supports detecting insufficient grounding and pruning documents for faster decoding. Evaluated on knowledge-based VQA, document VQA, and multimodal needle-in-a-haystack benchmarks.

Recommended citation: J. Chen, J. Mei, G. Yang, B. Byrne. "BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering." arXiv preprint arXiv:2604.22678.
Download Paper

According to Me: Long-Term Personalized Referential Memory QA

Published in arXiv preprint

We introduce ATM-Bench, a benchmark for long-term personalized referential memory question answering. Our work addresses the challenge of building AI systems that can maintain and reason over personalized memory across long-term interactions.

Recommended citation: J. Mei, J. Chen, G. Yang, X. Hou, M. Li, B. Byrne. "According to Me: Long-Term Personalized Referential Memory QA." arXiv preprint arXiv:2603.01990.
Download Paper

Patents

Apparatus and Method for Streaming Automatic Speech Recognition

Published in European Patent Office

A novel non-attention-based transformer architecture for efficient on-device streaming automatic speech recognition, achieving a 50% reduction in model size and computation along with a 60% decrease in end-to-end latency.

Recommended citation: J. Mei, Z. Zhang. "Apparatus and Method for Streaming Automatic Speech Recognition." EP Patent EP4404187A1, 2024.

Method, System, Medium, and Electronic Device for Optimizing Fault-Tolerant Neural Network Structure

Published in China National Intellectual Property Administration

A method for optimizing fault-tolerant neural network architectures using Bayesian optimization, addressing resistance variation and bit-flip in ReRAM devices.

Recommended citation: N. Ye, Z. Fang, J. Mei. "Method, System, Medium, and Electronic Device for Optimizing Fault-Tolerant Neural Network Structure." CN Patent CN113,570,056 A, 2021.