RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability

TL;DR

RaCoT tackles retrieval noise in RAG for long-tail and ambiguous questions by moving contrastive reasoning to the pre-retrieval stage. It builds a discriminative retrieval representation Q* through a contrastive triplet (Q_target, Q_contrast, Δ) via Δ-Prompting, and uses this to guide both retrieval and generation in a single pass. Across six QA benchmarks and multiple backbones, RaCoT demonstrates improved accuracy, robustness to adversarial distractors, and favorable efficiency relative to strong baselines. The work offers a practical path toward reliable, real-time QA in noisy, knowledge-sparse settings.

Abstract

Retrieval-Augmented Generation (RAG) faces a core bottleneck with knowledge-sparse and semantically ambiguous long-tail queries, where retrieval noise distorts reasoning and necessitates costly post-processing. To tackle this, we propose RaCoT (Retrieval-aware Contrastive-of-Thought), a novel framework that shifts contrastive thinking to the pre-retrieval stage. By automatically generating a semantically adjacent yet differently answered contrastive question and extracting a $Δ$-Prompt to capture their key differences, RaCoT guides the model to proactively focus on the ``critical details that determine answer divergence." This approach allows it to suppress semantic interference within a single retrieval pass, overcoming the theoretical bottleneck of single-vector queries that struggle to simultaneously encode signals for what to attend to and what to ignore. On six authoritative benchmarks, including PopQA and TriviaQA-unfiltered, RaCoT outperforms strong baselines like RankRAG and Self-RAG by 0.9-2.4 percentage points. It exhibits superior robustness, with a performance drop of only 8.6\% in adversarial tests, far surpassing the over 15\% degradation in other methods. Furthermore, its low latency (3.12s) and token overhead (11.54) place it on the accuracy-efficiency Pareto frontier, while ablation studies validate the necessity of each component. Ultimately, RaCoT reframes the RAG paradigm from ``post-hoc context cleaning" to ``a priori shaping of discriminative reasoning", offering an efficient and robust path toward reliable AI systems for real-time, resource-constrained deployments.

Figures (6)

• Figure 1: RankRAG improves retrieval quality by ranking results based on the original question, while our RaCoT generates contrastive questions to guide more reasoning-oriented retrieval.
• Figure 2: RaCoT enhances complex question answering by generating contrastive questions that differ in key semantics, retrieving evidence for both the original and contrastive questions, and forming fine-grained triplets. A value ranking network then scores and ranks candidate passages to select the most informative ones, improving retrieval-augmented generation with stronger semantic discrimination.
• Figure 3: The result of Retrieval Distractor Confusion shows that the green line indicates accuracy and the orange bars indicate distractor citation rate.
• Figure 4: RaCoT+ maintains accuracy under distraction, unlike other RAG methods.
• Figure 5: RaCoT+ achieves the highest accuracy with lower latency and token usage. Bubble size denotes answer accuracy, and color reflects average token consumption.