Sandwich Reasoning: An Answer-Reasoning-Answer Approach for Low-Latency Query Correction

TL;DR

SandwichR addresses the latency-accuracy trade-off in real-time query correction by outputting an initial correction $C_{init}$, a reasoning trajectory $R$, and a final correction $C_{final}$, enabling low-latency use while leveraging reasoning. It employs consistency-aware reinforcement learning with a margin-based rejection sampler to align $C_{init}$ with $C_{final}$ and distill reasoning into fast predictions. The approach is trained in two stages (SFT and RL) on a high-quality Multi-CPR-based dataset with injected errors, achieving state-of-the-art-like accuracy while delivering a 40-70% latency reduction compared to standard CoT methods. This work offers a practical pathway to deploy reasoning-enabled query correction in latency-constrained IR systems.

Abstract

Query correction is a critical entry point in modern search pipelines, demanding high accuracy strictly within real-time latency constraints. Chain-of-Thought (CoT) reasoning improves accuracy but incurs prohibitive latency for real-time query correction. A potential solution is to output an answer before reasoning to reduce latency; however, under autoregressive decoding, the early answer is independent of subsequent reasoning, preventing the model from leveraging its reasoning capability to improve accuracy. To address this issue, we propose Sandwich Reasoning (SandwichR), a novel approach that explicitly aligns a fast initial answer with post-hoc reasoning, enabling low-latency query correction without sacrificing reasoning-aware accuracy. SandwichR follows an Answer-Reasoning-Answer paradigm, producing an initial correction, an explicit reasoning process, and a final refined correction. To align the initial answer with post-reasoning insights, we design a consistency-aware reinforcement learning (RL) strategy: a dedicated consistency reward enforces alignment between the initial and final corrections, while margin-based rejection sampling prioritizes borderline samples where reasoning drives the most impactful corrective gains. Additionally, we construct a high-quality query correction dataset, addressing the lack of specialized benchmarks for complex query correction. Experimental results demonstrate that SandwichR achieves SOTA accuracy comparable to standard CoT while delivering a 40-70% latency reduction, resolving the latency-accuracy trade-off in online search.

Figures (4)

• Figure 1: Comparison of reasoning paradigms: traditional Chain-of-Thought (CoT) reasoning vs the proposed sandwich reasoning in this paper.
• Figure 2: Examples of three types of query errors including wrong words, missing words, disorder words.
• Figure 3: The overall workflow of the proposed SandwichR. It consists of two stages: (1) SandwichR Ability Acquisition via Supervised Fine-Tuning (SFT), and (2) Consistency-Aware Reinforcement Learning (RL).
• Figure 4: Performance Comparison of SandwichR with different RL training data sizes on E-commerce Dataset. The X-axis represents the RL training data size.