ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue

TL;DR

ACR tackles persistent challenges in multi-turn dialogue—contextual inertia and state drift—by introducing a modular Adaptive Context Refactoring framework. It uses a semantic Router to trigger context refactoring operators from a designed library and a Refactorer to rewrite history into a cleaner $\tilde{H}_t$, decoupling context management from reasoning. A Teacher-Guided Self-Evolving (TGSE) training loop progressively trains both components with a small supervision budget, achieving efficient, long-horizon reasoning. Experiments across seven QA benchmarks show consistent accuracy gains over strong baselines and substantial reductions in token usage compared with RL-based methods, demonstrating the practical value of active, need-driven history management for robust dialogue systems.

Abstract

Large Language Models (LLMs) have shown remarkable performance in multi-turn dialogue. However, in multi-turn dialogue, models still struggle to stay aligned with what has been established earlier, follow dependencies across many turns, and avoid drifting into incorrect facts as the interaction grows longer. Existing approaches primarily focus on extending the context window, introducing external memory, or applying context compression, yet these methods still face limitations such as \textbf{contextual inertia} and \textbf{state drift}. To address these challenges, we propose the \textbf{A}daptive \textbf{C}ontext \textbf{R}efactoring \textbf{(ACR)} Framework, which dynamically monitors and reshapes the interaction history to mitigate contextual inertia and state drift actively. ACR is built on a library of context refactoring operators and a teacher-guided self-evolving training paradigm that learns when to intervene and how to refactor, thereby decoupling context management from the reasoning process. Extensive experiments on multi-turn dialogue demonstrate that our method significantly outperforms existing baselines while reducing token consumption.

Figures (14)

• Figure 1: Illustration of two Challenges in multi-turn dialogue: (a) contextual inertia and (b) state drift.
• Figure 2: The framework of ACR. The pipeline includes Semantic Routing, where the Router selects the appropriate context operator, and Adaptive Refactoring, which refines the context for improved reasoning. The framework evolves through a Teacher-Guided Self-Evolution training, transitioning from supervised learning to autonomous decision-making, enhancing reasoning accuracy and efficiency.
• Figure 3: Comparison of average generated tokens across different methods.
• Figure 4: Training loss of the Router (LoRA) across TGSE rounds.
• Figure 5: Training loss of the Refactorer (LoRA) across TGSE rounds.