Text or Pixels? It Takes Half: On the Token Efficiency of Visual Text Inputs in Multimodal LLMs

TL;DR

This work investigates whether feeding an image of long text to multimodal LLMs can dramatically reduce input tokens without sacrificing performance. By rendering long passages as single images and processing them through a frozen vision encoder, the approach yields a fixed small set of visual tokens that replace large text spans, achieving a compression ratio ρ ≈ m/k. Across RULER long-context retrieval and CNN/DailyMail document summarization, the method maintains near-baseline accuracy while cutting decoder tokens by roughly 38–58% and, for larger models, delivering substantial end-to-end speedups. The technique, implemented via the ConTexImage pipeline, is model- and task-agnostic, requires no fine-tuning, and offers a practical, orthogonal path to scale long-context reasoning in LLMs.

Abstract

Large language models (LLMs) and their multimodal variants can now process visual inputs, including images of text. This raises an intriguing question: can we compress textual inputs by feeding them as images to reduce token usage while preserving performance? In this paper, we show that visual text representations are a practical and surprisingly effective form of input compression for decoder LLMs. We exploit the idea of rendering long text inputs as a single image and provide it directly to the model. This leads to dramatically reduced number of decoder tokens required, offering a new form of input compression. Through experiments on two distinct benchmarks RULER (long-context retrieval) and CNN/DailyMail (document summarization) we demonstrate that this text-as-image method yields substantial token savings (often nearly half) without degrading task performance.

Figures (8)

• Figure 1: Illustration of our text-as-image compression pipeline. Instead of feeding the entire 90-token context to the model (top), we convert the context into a single image and supply only the textual query alongside the image (bottom). The multimodal LLM (MLLM) reads the context from the image, so it processes just 50 visual tokens as input to the LLM decoder—cutting token usage by nearly half while still providing all the information needed to answer the question.
• Figure 2: Accuracy vs. text-token size ($m$) with fixed visual tokens ($k$) for GPT-4.1-mini (left) and Qwen2.5-VL-72B-Instruct (right). Each curve varies the rendered text length at a fixed $k$; vertical dashed lines mark $m=k$, where text and visual tokens are equal. Both models degrade as text density increases, though Qwen’s larger decoder sustains higher ratios before a sharper drop. The text-token tolerance (largest $m$ within 3 points of the text-only baseline) is shaded in the plots and reported in Table \ref{['tab:nihs_accuracy']}. Beyond these limits, accuracy falls rapidly, revealing the maximum achievable compression without loss.
• Figure 3: Text-token tolerance vs. visual token count. The maximum text tokens $m^\star$ that can be preserved without accuracy loss, plotted against the visual tokens $k$ generated from the image. Results show a consistent reduction of roughly $1/2$ in decoder tokens.
• Figure 4: Qwen2.5-VL-7B-Instruct: accuracy vs. text-token size ($m$) with fixed visual tokens ($k$). Each curve varies the amount of rendered text for a fixed visual token size. Compared to the 72B version (Figure \ref{['fig:qwen_curve_sub']}), the smaller 7B model exhibits a much steeper performance degradation as text density increases, indicating that model scale is a critical factor for effective visual text processing.
• Figure 5: Rendered image input for the Ruler task at context length 1500 ($600\times800$ image resolution). Here there is almost no accuracy degradation. This example illustrates how textual sequences are converted into rasterized images for multimodal processing.