Facilitating Generative Retrieval with Logical Denoising for Interpretable Conversational Search

📖 Overview

Conversational search allows multi-turn user-system interactions to support complex information seeking. It is challenging due to frequent topic shifts and ambiguous intentions in conversation. Although existing methods attempt to optimize retrieval performance through query rewriting or session encoding, they still face crucial challenges, including (1) Noisy context: the diverse types of noises in the conversation context prevent models from achieving reliable and consistent contextual understanding; (2) Poor interpretability: the lack of transparency in how results are generated undermines user trust, hindering the practical application of conversational search systems. In this paper, we propose LogiCGR, a novel framework that utilizes curriculum learning and group relative policy optimization (GRPO) to perform logic-enhanced retrieval, improving the robustness and interpretability of conversational search. Specifically, LogiCGR equips large language models (LLMs) with logical denoising and generative retrieval, integrating them seamlessly through an adaptive framework. Additionally, we introduce a lightweight module that works with generative retrieval for self-dual-path retrieval, thus delivering complementary performance gains. Extensive experiments and intuitive case studies demonstrate that our proposed LogiCGR outperforms state-of-the-art baselines in both retrieval performance and interpretability.

⚙️ Environment

Our LogiCGR involves three stages: Logic-aware Retrieval Cold Start, Retrieval-driven Logic Exploration and Logical-enhanced Generative Retrieval.

1. Install requirements:

pip install -r requirements.txt

2. Install SEAL (for constraint index):

git clone --recursive https://github.com/facebookresearch/SEAL.git
cd SEAL
env CFLAGS='-fPIC' CXXFLAGS='-fPIC' res/external/sdsl-lite/install.sh
pip install -r requirements.txt
pip install -r requirements_extra.txt
pip install -e .

SEAL need some patches for compatibility with newer transformers versions:

Edit seal/beam_search.py:

• Change from transformers.generation_utils import xxx to from transformers.generation.utils import xxx
• Change from transformers.generation_logits_process import TopKLogitsWarper to from transformers.utils.dummy_pt_objects import TopKLogitsWarper

Logic-aware Retrieval Cold Start uses the official environment of LLaMA-Factory. For environment installation, please refer to the LLaMA-Factory.

Retrieval-driven Logic Exploration uses the official environment of verl. For environment installation, please refer to the verl.

📁 Data

We use the data processed by GCoQA. Meanwhile, we provide the data we processed for training.

1. Download official data from GCoQA and put them into data/gcoqa folder.
2. Our processed data can be obtained from data folder. data/grc includes the training data used in Generative Retrieval Cognition stage, and data/ldc includes the training data used in Logical Denoising Cognition stage. Both of which have been processed into the format required by LLaMA-Factory.

💡 Logic-aware Retrieval Cold Start

Stage 1: Generative Retrieval Cognition

1. Copy data/grc to LLaMA-Factory/data. Paste the following content into data/dataset_info.json to register the data (taking the TopiOCQA as example, the same below).

"logicgr_topiocqa_grc": {
    "file_name": "grc/topiocqa_grc.json"
  },

2. Run the SFT stage 1 training script.

conda activate llama_factory

export CUDA_VISIBLE_DEVICES=0,1
export WANDB_API_KEY="YOUR_WANDB_KEY"

PROJECT_NAME=llama32_3b_topiocqa
EXPERIMENT_NAME=logicgr_sft_stage_1

llamafactory-cli train \
    --model_name_or_path meta-llama/Llama-3.2-3B-Instruct \
    --trust_remote_code \
    --stage sft \
    --do_train \
    --finetuning_type full \
    --dataset logicgr_topiocqa_grc \
    --template llama3 \
    --cutoff_len 2048 \
    --max_samples 50000 \
    --overwrite_cache \
    --preprocessing_num_workers 16 \
    --dataloader_num_workers 4 \
    --output_dir LLaMA-Factory/checkpoints/$PROJECT_NAME/$EXPERIMENT_NAME \
    --logging_steps 3 \
    --save_steps 500 \
    --plot_loss \
    --overwrite_output_dir \
    --save_only_model true \
    --report_to wandb \
    --per_device_train_batch_size 8 \
    --gradient_accumulation_steps 4 \
    --learning_rate 1e-5 \
    --num_train_epochs 1.0 \
    --lr_scheduler_type cosine \
    --warmup_ratio 0.1 \
    --seed 42 \
    --bf16 \
    --ddp_timeout 180000000 \
    --deepspeed examples/deepspeed/ds_z2_config.json

All the models trained by LLaMA-Factory should be stored in LLaMA-Factory/checkpoints (the same below).

Stage 2: Logical Denoising Cognition

1. Generate high-quality logical denoising Chains-of-Thoughts (CoTs).

python cot.py \
  --load_small False \
  --test_file_path ./data/topiocqa/topiocqa_dev.json \
  --output_file_dir ./data/ldc \
  --overwrite_output True \
  --seed 42 \

However, the generated data needs to be converted into the form of LLaMA-Factory. We provide the formatted data in data/ldc.

2. Register the data into data/dataset_info.json. Use the same operation metioned in stage Generative Retrieval Cognition.

3. Run the SFT stage 2 training script.

conda activate llama_factory

export CUDA_VISIBLE_DEVICES=0,1
export WANDB_API_KEY="YOUR_WANDB_KEY"

PROJECT_NAME=llama32_3b_topiocqa
EXPERIMENT_NAME=logicgr_sft_stage_2

llamafactory-cli train \
    --model_name_or_path LLaMA-Factory/checkpoints/$PROJECT_NAME/logicgr_sft_stage_1 \
    --trust_remote_code \
    --stage sft \
    --do_train \
    --finetuning_type full \
    --dataset logicgr_topiocqa_ldc \
    --template llama3 \
    --cutoff_len 2048 \
    --max_samples 50000 \
    --overwrite_cache \
    --preprocessing_num_workers 16 \
    --dataloader_num_workers 4 \
    --output_dir LLaMA-Factory/checkpoints/$PROJECT_NAME/$EXPERIMENT_NAME \
    --logging_steps 3 \
    --save_steps 500 \
    --plot_loss \
    --overwrite_output_dir \
    --save_only_model true \
    --report_to wandb \
    --per_device_train_batch_size 8 \
    --gradient_accumulation_steps 4 \
    --learning_rate 1e-5 \
    --num_train_epochs 2.0 \
    --lr_scheduler_type cosine \
    --warmup_ratio 0.1 \
    --seed 42 \
    --bf16 \
    --ddp_timeout 180000000 \
    --deepspeed examples/deepspeed/ds_z2_config.json

🏆 Retrieval-driven Logic Exploration

1. Run verl/logicgr_preprocess_data.py to process the data into the .parquet format required by verl.

2. Run the GRPO training script.

conda activate verl

#!/bin/bash
set -x

export CUDA_VISIBLE_DEVICES=0,1
export WANDB_API_KEY="YOUR_WANDB_KEY"

PROJECT_NAME=llama32_3b_topiocqa
EXPERIMENT_NAME=logicgr_grpo

python3 -m verl.trainer.main_ppo \
    algorithm.adv_estimator=grpo \
    data.train_files=../data/topiocqa/train.parquet \
    data.val_files=../data/topiocqa/test.parquet \
    data.train_batch_size=128 \
    data.max_prompt_length=2048 \
    data.max_response_length=512 \
    data.filter_overlong_prompts=True \
    data.truncation='error' \
    actor_rollout_ref.model.path=../LLaMA-Factory/checkpoints/llama32_3b_topiocqa/logicgr_sft_stage_2 \
    actor_rollout_ref.model.use_shm=True \
    actor_rollout_ref.actor.optim.lr=1e-6 \
    actor_rollout_ref.actor.optim.lr_warmup_steps=30 \
    actor_rollout_ref.model.use_remove_padding=True \
    actor_rollout_ref.actor.ppo_mini_batch_size=128 \
    actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=32 \
    actor_rollout_ref.actor.use_kl_loss=True \
    actor_rollout_ref.actor.kl_loss_coef=0.001 \
    actor_rollout_ref.actor.kl_loss_type=low_var_kl \
    actor_rollout_ref.actor.entropy_coeff=0 \
    actor_rollout_ref.model.enable_gradient_checkpointing=True \
    actor_rollout_ref.actor.fsdp_config.param_offload=False \
    actor_rollout_ref.actor.fsdp_config.optimizer_offload=False \
    actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=64 \
    actor_rollout_ref.rollout.tensor_model_parallel_size=1 \
    actor_rollout_ref.rollout.name=vllm \
    actor_rollout_ref.rollout.gpu_memory_utilization=0.6 \
    actor_rollout_ref.rollout.n=6 \
    actor_rollout_ref.rollout.load_format=safetensors \
    actor_rollout_ref.rollout.layered_summon=True \
    actor_rollout_ref.rollout.temperature=0.8 \
    actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=64 \
    actor_rollout_ref.ref.fsdp_config.param_offload=True \
    algorithm.kl_ctrl.kl_coef=0.001 \
    trainer.critic_warmup=0 \
    trainer.logger='["console", "tensorboard", "wandb"]' \
    trainer.n_gpus_per_node=2 \
    trainer.nnodes=1 \
    trainer.save_freq=100 \
    trainer.test_freq=-1 \
    trainer.total_epochs=1 \
    trainer.val_before_train=False \
    trainer.project_name=$PROJECT_NAME \
    trainer.experiment_name=$EXPERIMENT_NAME \
    trainer.default_local_dir=checkpoints/$PROJECT_NAME/$EXPERIMENT_NAME \
    custom_reward_function.path=logicgr_reward_function.py \
    custom_reward_function.name=compute_score \
    $@ 2>&1 | tee logs/${PROJECT_NAME}_${EXPERIMENT_NAME}.log

All the models trained by verl should be stored in folder verl/checkpoints.

🔍 Logical-enhanced Generative Retrieval

Step 1: Constraint Index Adaptation

Run the script below to build constraint index.

FILE_I=./data/gcoqa/full_wiki_segments.json
FILE_O=./index/llama32_3b_trie_begin_with_space_end_with_retrieve.fm_index

python build_trie.py \
    --input_file_path $FILE_I \
    --output_file_path $FILE_O \
    --hf_model meta-llama/Llama-3.2-3B-Instruct \
    --jobs 40 \

Step 2: Inference Framework Adaptation

Run the script below to obtain LogiCGR results.

export CUDA_VISIBLE_DEVICES=0,1

python logicgr.py \
  --load_small False \
  --model_name_or_path ./verl/checkpoints/llama32_3b_topiocqa/logicgr_grpo \
  --test_file_path ./data/topiocqa/topiocqa_test.json \
  --trie_file_path ./index/llama32_3b_trie_begin_with_space_end_with_retrieve.fm_index \
  --output_file_dir ./outputs/llama32_3b_topiocqa/logicgr \
  --overwrite_output True \
  --second_stage_num_beams 5 \
  --second_stage_top_k 5 \
  --seed 42 \
  --forced_bos_token_id 128000

Step 3: Self-dual-path Retrieval

1. We need ANCE for Self-dual-path Retrieval stage. First, download the retriever ANCE. Then, put the model into retrieve/ance.
2. Run the script below to construct dense index.

export CUDA_VISIBLE_DEVICES=0,1

RETRIEVER_PATH="./retrieve/ance"

torchrun --nproc_per_node=8 \
  --master_port 22004 \
  indexing.py \
  --model_name_or_path $RETRIEVER_PATH \
  --model_type "ance" \
  --normalize_emb \
  --max_p_len 384 \
  --per_device_eval_batch_size 64 \
  --num_psg_per_block 1000000 \
  --corpus_path ./data/gcoqa/full_wiki_segments.json \
  --data_output_dir ./index/ance_index \
  --force_emptying_dir

3. Run the script below to obtain denoising process retrieval results.

export CUDA_VISIBLE_DEVICES=0,1

RETRIEVER_PATH="./retrieve/ance"

python retrieve.py \
  --model_name_or_path $RETRIEVER_PATH \
  --model_type "ance" \
  --normalize_emb \
  --index_dir ./index/ance_index \
  --query_data_path_list ./outputs/llama32_3b_topiocqa/logicgr/retrieval_results_for_sr_retrieve.json \
  --max_q_len 512 \
  --embedding_size 768 \
  --data_output_dir ./outputs/llama32_3b_topiocqa/logicgr_sr/retrieval_results.json \
  --top_n=100 \
  --force_emptying_dir \

4. Run the script below to obtain Self-dual-path Retrieval ranking results.

GENERATIVE_RETRIEVAL_PATH="./outputs/llama32_3b_topiocqa/logicgr/retrieval_results.json"
DENSE_RETRIEVAL_PATH="./outputs/llama32_3b_topiocqa/logicgr_sr/retrieval_results.json"
OUTPUT_RANKING_PATH="./outputs/llama32_3b_topiocqa/logicgr_sr/ranking_results.json"

python merge_ranking.py \
  --generative_retrieval_path $GENERATIVE_RETRIEVAL_PATH \
  --dense_retrieval_path $DENSE_RETRIEVAL_PATH \
  --output_file_path $OUTPUT_RANKING_PATH \

📊 Evaluation

Modify the retrieval result path in evaluate.py, and run evaluate.py to obtain the final LogiCGR or LogiCGR-SR results.

🔭 Citation

If you find this work helpful, please cite our paper:

@inproceedings{liu2026facilitating,
  title={Facilitating Generative Retrieval with Logical Denoising for Interpretable Conversational Search},
  author={Liu, Qichuan and Zhang, Chentao and Hu, Yuxuan and Zheng, Chenfeng and Zhang, Qinggang and Zhang, Zhihong},
  booktitle={Proceedings of the ACM Web Conference 2026},
  pages={2296--2307},
  year={2026}
}