ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model

TL;DR

This work tackles the inefficiency of long Chain-of-Thought (CoT) reasoning in large language models by introducing ImgCoT, a framework that compresses CoT into compact visual tokens rather than text. By rendering CoT as images and learning latent visual tokens with a spatial inductive bias, ImgCoT better preserves global reasoning structure while discarding surface linguistic details. A hybrid variant, Loose ImgCoT (L-ImgCoT), adds a small set of textual reasoning steps selected by a gamma-based threshold to retain essential domain-specific details under tighter inference budgets. Across multiple datasets and backbones, ImgCoT and L-ImgCoT outperform text-based latent compression baselines and offer substantial gains in reasoning efficiency with competitive or superior accuracy, demonstrating the practical value of visual CoT compression.

Abstract

Compressing long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs). Recent studies employ autoencoders to achieve this by reconstructing textual CoT from latent tokens, thus encoding CoT semantics. However, treating textual CoT as the reconstruction target forces latent tokens to preserve surface-level linguistic features (e.g., word choice and syntax), introducing a strong linguistic inductive bias that prioritizes linguistic form over reasoning structure and limits logical abstraction. Thus, we propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images. This substitutes linguistic bias with spatial inductive bias, i.e., a tendency to model spatial layouts of the reasoning steps in visual CoT, enabling latent tokens to better capture global reasoning structure. Moreover, although visual latent tokens encode abstract reasoning structure, they may blur reasoning details. We thus propose a loose ImgCoT, a hybrid reasoning that augments visual latent tokens with a few key textual reasoning steps, selected based on low token log-likelihood. This design allows LLMs to retain both global reasoning structure and fine-grained reasoning details with fewer tokens than the complete CoT. Extensive experiments across multiple datasets and LLMs demonstrate the effectiveness of the two versions of ImgCoT.

Figures (8)

• Figure 1: Comparison between three different latent reasoning methods. (a) The reconstructed textual CoT is linguistically fluent but contains structural errors compared with the original CoT, revealing that latent tokens in text-based compression tend to memorize expression patterns rather than reasoning organization. (b) The well-structured CoT in reconstructed visual CoT reflects that latent tokens in image-based compression preserve the global reasoning structure but blur the local reasoning details. (c) Loose ImgCoT further allocates a small number of additional tokens to allow LLMs to explicitly generate critical reasoning steps under uncertainty, thereby combining global reasoning structure with local reasoning precision.
• Figure 2: The overall training pipeline of ImgCoT. First, a visual encoder is trained by reconstructing visualized text, thereby compressing textual inputs into latent tokens. Then, the LLM is trained in an autoregressive manner to perform reasoning over the visual latent tokens. $\langle\cdot \rangle$ denotes the output sequence processed by the LLM tokenizer.
• Figure 3: Qualitative comparison between the original CoT and its reconstructions from different latent representations. Text-based compression preserves surface-level linguistic forms, whereas image-based compression captures the global reasoning skeleton and stepwise layout of CoT. These examples illustrate the intrinsic inductive biases of different latent representations.
• Figure 4: Performance comparison under varying numbers of visual versus textual latent tokens. The LLM is Qwen2.5-0.5B-Instruction.
• Figure 5: Effectiveness of Retaining Critical Reasoning Steps. The strikethrough text in (b) corresponds to reasoning deemed unimportant and filtered out by our strategy. $[\cdot]$ represents the latent tokens. The LLM adopted here is Qwen2.5-0.5B-Instruction.