Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation

TL;DR

This work shows that how retrieved content is presented—its surface format—has a profound impact on long-context reasoning in retrieval-augmented generation. It identifies tokenization and attention attribution as key mechanisms driving format sensitivity and introduces Contextual Normalization (C-Norm), a training-free, model-aware pipeline that adaptively normalizes context formatting using an Attention Balance Score. Across controlled and real-world benchmarks, C-Norm consistently improves robustness to order variation and enhances long-context utilization, achieving substantial gains especially in challenging long-context settings. The findings highlight the practical importance of input formatting in RAG, offering a simple yet effective method to stabilize and scale long-context reasoning.

Abstract

Retrieval-Augmented Generation (RAG) has become an essential approach for extending the reasoning and knowledge capacity of large language models (LLMs). While prior research has primarily focused on retrieval quality and prompting strategies, the influence of how the retrieved documents are framed, i.e., context format, remains underexplored. We show that seemingly superficial choices, such as delimiters or structural markers in key-value extraction, can induce substantial shifts in accuracy and stability, even when semantic content is identical. To systematically investigate this effect, we design controlled experiments that vary context density, delimiter styles, and positional placement, revealing the underlying factors that govern performance differences. Building on these insights, we introduce Contextual Normalization, a lightweight strategy that adaptively standardizes context representations before generation. Extensive experiments on both controlled and real-world RAG benchmarks across diverse settings demonstrate that the proposed strategy consistently improves robustness to order variation and strengthens long-context utilization. These findings underscore that reliable RAG depends not only on retrieving the right content, but also on how that content is presented, offering both new empirical evidence and a practical technique for better long-context reasoning.

Figures (6)

• Figure 1: Illustration of the fact that different formats yield substantial differences in RAG performance.
• Figure 2: Model performance on key-value extraction task.
• Figure 3: Qwen2.5-1.5B performance across delimiter configurations. Across settings, we observe a negative trend: configurations that inflate tokenization length tend to yield lower OAA.
• Figure 4: Attention attributions in long-context reasoning under low-density settings. The x-axis denotes the position of input tokens.
• Figure 5: Overview of the proposed C-Norm pipeline.