Drift No More? Context Equilibria in Multi-Turn LLM Interactions

TL;DR

This work addresses how context drift unfolds in sustained multi-turn LLM interactions. It introduces a simple, principled dynamical framework that treats drift as a bounded stochastic recurrence, quantified by the turn-wise KL divergence $D_t = D_{\mathrm{KL}}(q_t \;||\; p_t)$ between a test model and a goal-consistent reference. Through synthetic tasks and realistic $\tau$-Bench simulations, the authors show that drift stabilizes at finite equilibria and that lightweight reminder interventions reliably lower the equilibrium divergence in line with the proposed model. The results suggest that multi-turn drift is a controllable equilibrium phenomenon rather than an inexorable decay, with practical implications for designing robust, long-horizon interactive AI systems.

Abstract

Large Language Models (LLMs) excel at single-turn tasks such as instruction following and summarization, yet real-world deployments require sustained multi-turn interactions where user goals and conversational context persist and evolve. A recurring challenge in this setting is context drift: the gradual divergence of a model's outputs from goal-consistent behavior across turns. Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics. In this work, we present a study of context drift in multi-turn interactions and propose a simple dynamical framework to interpret its behavior. We formalize drift as the turn-wise KL divergence between the token-level predictive distributions of the test model and a goal-consistent reference model, and propose a recurrence model that interprets its evolution as a bounded stochastic process with restoring forces and controllable interventions. We instantiate this framework in both synthetic long-horizon rewriting tasks and realistic user-agent simulations such as in $τ$-Bench, measuring drift for several open-weight LLMs that are used as user simulators. Our experiments consistently reveal stable, noise-limited equilibria rather than runaway degradation, and demonstrate that simple reminder interventions reliably reduce divergence in line with theoretical predictions. Together, these results suggest that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay, providing a foundation for studying and mitigating context drift in extended interactions.

Figures (8)

• Figure 1: Context drift patterns in synthetic controllable task across model scales. Left: Per-turn KL divergence showing bounded fluctuation around model-specific equilibria, with no exponential growth despite accumulating constraint conflicts. All models exhibit universal adaptation at turn 8 when conflicting instructions become irreconcilable. Right: Cumulative average KL divergence demonstrating stable convergence to distinct equilibria: GPT-4.1 ($D^* \approx 0.7$), LLaMA-3.1-70B ($D^* \approx 15.0$), and LLaMA-3.1-8B ($D^* \approx 17.5$).
• Figure 2: KL divergence trajectories without reminder interventions.
• Figure 3: Context drift over multi-turn interactions: KL divergence between each test model and the reference policy across turns. Solid lines indicate the baseline setting without interventions, while dashed lines indicate the reminder setting with explicit goal reminders injected at turns $t=4$ and $t=7$. Shaded regions denote $\pm$ standard error. Models compared: LLaMA 3.1 8B (blue), Qwen 2 7B Instruct (orange), and LLaMA 3.1 70B (green). Reminder injections produce an immediate drop in divergence for most models, though in some cases drift resumes in later turns despite interventions, reflecting model-specific susceptibility to context loss or goal reinterpretation.
• Figure 4: Synthetic Task Setup
• Figure 5: Instructions for Synthetic Task: Academic Writing Assistant