Read As Human: Compressing Context via Parallelizable Close Reading and Skimming

TL;DR

This work tackles the bottleneck of long-context processing in LLMs by introducing RAM, Read As HuMan, a context compression framework that reads highly relevant segments in full while skimming the rest through query-guided compression. RAM encodes all segments and the query in parallel, builds a hybrid representation by concatenating verbatim text with compact skim vectors, and uses a contrastive objective to sharpen the boundary between close reading and skimming. It achieves state-of-the-art or competitive results on multiple QA and summarization benchmarks with up to ≈12x end-to-end speedups on long inputs, and demonstrates robustness across backbones and compression rates. The approach offers substantial efficiency gains without sacrificing interpretability, enabling practical deployment of LLMs in long-context settings.

Abstract

Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computational inefficiency and redundant information. We propose RAM (Read As HuMan), a context compression framework that adopts an adaptive hybrid reading strategy, to address these challenges. Inspired by human reading behavior (i.e., close reading important content while skimming less relevant content), RAM partitions the context into segments and encodes them with the input query in parallel. High-relevance segments are fully retained (close reading), while low-relevance ones are query-guided compressed into compact summary vectors (skimming). Both explicit textual segments and implicit summary vectors are concatenated and fed into decoder to achieve both superior performance and natural language format interpretability. To refine the decision boundary between close reading and skimming, we further introduce a contrastive learning objective based on positive and negative query-segment pairs. Experiments demonstrate that RAM outperforms existing baselines on multiple question answering and summarization benchmarks across two backbones, while delivering up to a 12x end-to-end speedup on long inputs (average length 16K; maximum length 32K).

Figures (5)

• Figure 1: Comparison of RAM with other task-aware context compression methods. Existing task-aware methods either require loading the entire input sequence at once for compression (Figure \ref{['fig:a']}) or rely on autoregressive compression (Figure \ref{['fig:b']}), both of which suffer from computational inefficiency. In contrast, RAM processes all segments and the query in parallel and adaptively decides (based on relevance) which segments to close reading and which to skim (Figure \ref{['fig:c']}).
• Figure 2: Overview of the RAM framework. The framework consists of two main stages: (1) Query-Aware Parallel Encoding: The query and segmented context are encoded in parallel. A query-guided attention mechanism computes a relevance score for each segment, determining whether to retain it as original text (close reading) or compress it into a compact vector (skimming), with the number of segments to retain derived from the compression ratio via Eq. \ref{['eq:k_def']}. (2) Adaptive Compression and Training: The retained text and compressed vectors are fed into a decoder to produce the final output. The model is trained end-to-end using a language modeling objective $\mathcal{L}_{\text{nll}}$ and a contrastive learning objective $\mathcal{L}_{\text{con}}$ to jointly optimize overall performance.
• Figure 3: Performance under different compression rates on NarrativeQA. All methods use Qwen3-4B as the backbone.
• Figure 4: A case study from 2WikiMQA dataset.
• Figure 5: Prompt template of annotation used by Qwen3-235B-A22B-Instruct.