VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning

TL;DR

This work tackles the efficiency bottleneck of long-context reasoning in LLMs by introducing VTC-R1, a paradigm that renders intermediate reasoning steps as compact images and uses them as optical memory within vision-language models. By iteratively generating reasoning segments and compressing prior steps into images, the approach achieves about $3$–$4\\times$ token compression and up to $2.7\\times$ end-to-end latency improvements without additional training or external summarization models. Trained on a large OpenR1-Math-Inf-derived dataset and evaluated on GSM8K, MATH500, AIME25, AMC23, and GPQA-Diamond, VTC-R1 consistently improves accuracy over standard long-context reasoning and shows strong generalization to out-of-distribution tasks. The method also delivers efficiency gains through lightweight rendering (roughly 0.12s per image for about 1,600 text tokens) and a streamlined batch-inference pipeline, indicating practical potential for scalable reasoning-intensive applications.

Abstract

Long-context reasoning has significantly empowered large language models (LLMs) to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we propose VTC-R1, a new efficient reasoning paradigm that integrates vision-text compression into the reasoning process. Instead of processing lengthy textual traces, VTC-R1 renders intermediate reasoning segments into compact images, which are iteratively fed back into vision-language models as "optical memory." We construct a training dataset based on OpenR1-Math-220K achieving 3.4x token compression and fine-tune representative VLMs-Glyph and Qwen3-VL. Extensive experiments on benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context reasoning. Furthermore, our approach significantly improves inference efficiency, achieving 2.7x speedup in end-to-end latency, highlighting its potential as a scalable solution for reasoning-intensive applications. Our code is available at https://github.com/w-yibo/VTC-R1.

Figures (10)

• Figure 1: Comparison between existing efficient reasoning approaches and vision-text compression (VTC). Existing methods either require additional training or sampling procedures, or rely on external strong models. In contrast, VTC leverages lightweight rendering to transform long textual reasoning traces into compact visual representations, enabling VLMs to encode information with significantly fewer vision tokens (3-4× compression). This approach is both lightweight and model-free.
• Figure 2: Comparison between standard long-context reasoning and the proposed VTC-R1 reasoning paradigm. (a) Standard long-context reasoning processes the entire reasoning trace as a single long sequence, leading to increasing computational and memory costs as the context grows. (b) VTC-R1 reformulates long-context reasoning as an iterative process. At each iteration, the current reasoning segment is generated and the preceding segments are rendered into compact images, which are fed back to the model together with the original question. These rendered images function as a form of optical memory, enabling efficient multi-step reasoning with reduced token usage.
• Figure 3: Distribution of data index. The index indicates the order of a reasoning segment for a given problem, where index $0$ corresponds to the first segment. Most samples terminate at early steps, while a small fraction requires more than four iterations.
• Figure 4: Accuracy of the proposed method across benchmarks under different maximum iteration epochs. The epoch index denotes the maximum number of allowed reasoning iterations, and predictions that terminate earlier are also included in evaluation. The dashed line indicates the single-round baseline (standard long-context reasoning for 8192 maximum tokens).
• Figure 5: Default rendering configuration used in our experiments.