Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DZ-TDPO: Non-Destructive Temporal Alignment for Mutable State Tracking in Long-Context Dialogue

Yijun Liao

TL;DR

This work addresses State Inertia in long-context dialogues by identifying Temporal Attention Imbalance as the root of outdated-history overfitting. It introduces DZ-TDPO, a non-destructive framework that combines a dynamic, time-aware TDPO-DKL optimization with a Dual-Zone Temporal Attention structure that separates immutable system prompts from mutable history. Empirical results on MSC and UltraChat show state-of-the-art conflict resolution with minimal perplexity overhead, and a scaling analysis reveals a Capacity-Stability Trade-off where larger models resist temporal biases more robustly. The approach offers a practical, safe, and scalable path for long-context agents to update beliefs and preferences without sacrificing general linguistic capabilities.

Abstract

Long-context dialogue systems suffer from State Inertia, where static constraints prevent models from resolving conflicts between evolving user intents and established historical context. To address this, we propose DZ-TDPO, a non-destructive alignment framework that synergizes conflict-aware dynamic KL constraints with a calibrated temporal attention bias. Experiments on the Multi-Session Chat (MSC) dataset demonstrate that DZ-TDPO achieves state-of-the-art win rates (55.4% on Phi-3.5) while maintaining robust zero-shot generalization. Our scaling analysis reveals a "Capacity-Stability Trade-off": while smaller models incur an "alignment tax" (perplexity surge) to overcome historical inertia, the larger Qwen2.5-7B model achieves 50.8% win rate with negligible perplexity overhead. This confirms that TAI can be alleviated via precise attention regulation rather than destructive weight updates, preserving general capabilities (MMLU) across model scales. Code and data are available: https://github.com/lyj20071013/DZ-TDPO

DZ-TDPO: Non-Destructive Temporal Alignment for Mutable State Tracking in Long-Context Dialogue

TL;DR

This work addresses State Inertia in long-context dialogues by identifying Temporal Attention Imbalance as the root of outdated-history overfitting. It introduces DZ-TDPO, a non-destructive framework that combines a dynamic, time-aware TDPO-DKL optimization with a Dual-Zone Temporal Attention structure that separates immutable system prompts from mutable history. Empirical results on MSC and UltraChat show state-of-the-art conflict resolution with minimal perplexity overhead, and a scaling analysis reveals a Capacity-Stability Trade-off where larger models resist temporal biases more robustly. The approach offers a practical, safe, and scalable path for long-context agents to update beliefs and preferences without sacrificing general linguistic capabilities.

Abstract

Long-context dialogue systems suffer from State Inertia, where static constraints prevent models from resolving conflicts between evolving user intents and established historical context. To address this, we propose DZ-TDPO, a non-destructive alignment framework that synergizes conflict-aware dynamic KL constraints with a calibrated temporal attention bias. Experiments on the Multi-Session Chat (MSC) dataset demonstrate that DZ-TDPO achieves state-of-the-art win rates (55.4% on Phi-3.5) while maintaining robust zero-shot generalization. Our scaling analysis reveals a "Capacity-Stability Trade-off": while smaller models incur an "alignment tax" (perplexity surge) to overcome historical inertia, the larger Qwen2.5-7B model achieves 50.8% win rate with negligible perplexity overhead. This confirms that TAI can be alleviated via precise attention regulation rather than destructive weight updates, preserving general capabilities (MMLU) across model scales. Code and data are available: https://github.com/lyj20071013/DZ-TDPO

Paper Structure

This paper contains 60 sections, 40 equations, 3 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Related Work
  4. Preference Alignment
  5. Long-Context Attention Mechanisms
  6. Temporal Modeling in Dialogue
  7. Problem Formulation
  8. Methodology
  9. Optimization Level: TDPO-DKL
  10. Representation Level: Dual-Zone Temporal Attention
  11. Experiments
  12. Experimental Setup
  13. Datasets.
  14. Baselines
  15. Results
  16. ...and 45 more sections

Figures (3)

  • Figure 1: The dynamic mechanism of TDPO-DKL. As the dialogue progresses towards the current turn $T$, the KL coefficient $\beta(t;T)$ (blue solid line) decreases to relax constraints, while the temporal weight $w(t;T)$ (orange dashed line) increases to amplify the gradient signal for recent updates.
  • Figure 2: Illustration of Stability Dynamics. The plot visualizes the divergence in perplexity. Standard DPO (Red) exhibits a characteristic distribution collapse, where the rigid historical constraints force the model to degrade its linguistic capabilities to satisfy the reward ($PPL \rightarrow 102.3$). In contrast, DZ-TDPO (Blue) maintains a stable trajectory ($PPL \rightarrow 26.0$), confirming that our structural bias effectively decouples conflict resolution from general linguistic modeling.
  • Figure 3: Visualization of the Structural Attention Bias. The plot compares the attention logit bias of the standard decay ($\lambda=0.5$, gray dashed) versus a hypothetical aggressive decay ($\lambda=0.68$, red solid). The steeper slope of the hypothetical curve illustrates the stronger penalty required if we were to prioritize conflict resolution over stability; however, our fixed prior ($\lambda=0.5$) strikes a balanced trade-off, effectively suppressing outdated information on distant history to resolve temporal conflicts.