See the Text: From Tokenization to Visual Reading

TL;DR

This work introduces SeeTok, a vision-centric tokenization that renders text as images and leverages pretrained multimodal LLMs to process textual content, challenging the dominance of subword tokenization. By using a vision encoder and a lightweight projector, SeeTok compresses text into visual tokens and, with LoRA-based instruction tuning, enables effective visual-text instruction following without retraining from scratch. Across language understanding and multilingual translation tasks, SeeTok achieves competitive or superior performance while delivering ~$4\times$ fewer tokens and a ~$70.5\%$ reduction in FLOPs, and it demonstrates stronger cross-lingual transfer and robustness to typographic noise. The approach generalizes across different MLLMs and points toward a cognitively inspired, fully multimodal future for language models that can better handle diverse scripts and noisy inputs.

Abstract

People see text. Humans read by recognizing words as visual objects, including their shapes, layouts, and patterns, before connecting them to meaning, which enables us to handle typos, distorted fonts, and various scripts effectively. Modern large language models (LLMs), however, rely on subword tokenization, fragmenting text into pieces from a fixed vocabulary. While effective for high-resource languages, this approach over-segments low-resource languages, yielding long, linguistically meaningless sequences and inflating computation. In this work, we challenge this entrenched paradigm and move toward a vision-centric alternative. Our method, SeeTok, renders text as images (visual-text) and leverages pretrained multimodal LLMs to interpret them, reusing strong OCR and text-vision alignment abilities learned from large-scale multimodal training. Across three different language tasks, SeeTok matches or surpasses subword tokenizers while requiring 4.43 times fewer tokens and reducing FLOPs by 70.5%, with additional gains in cross-lingual generalization, robustness to typographic noise, and linguistic hierarchy. SeeTok signals a shift from symbolic tokenization to human-like visual reading, and takes a step toward more natural and cognitively inspired language models.

Figures (12)

• Figure 1: Left: Reading proceeds through a visual–linguistic pathway: the visual stream identifies letter shapes and patterns in the visual cortex and packages them into recognizable word forms via the visual word form area; the linguistic stream in the left-hemisphere derives meaning. Right: Subword tokenization tends to over-segment low-resource languages due to insufficient vocabulary coverage, e.g., a 2-word Kyrgyz phrase ("world knowledge") is split into 11 text tokens. Our vision-centric tokenization instead compresses the phrase into a single visual token by aggregating features from four adjacent image patches through the projector.
• Figure 2: Overview of SeeTok. Text is rendered into an image, processed by the vision-centric tokenization, and fed to the LLM. LoRA layers further boost its ability to follow visual-text instructions.
• Figure 3: Left: Fertility and token compression ratio across low-resource languages, comparing text and vision-centric tokenization. Right: COMET-22 scores on FLORES for translations from low-resource languages into English, comparing Qwen2.5-VL 3B trained with visual-text input and with pure-text input.
• Figure 4: Accuracy drop on MMLU hendrycksmeasuring under different orthographic perturbations (character-, visual-, and word-level noise). The vision tokenization–based model (blue) shows markedly smaller declines than the text-tokenization counterpart (orange), demonstrating stronger robustness to surface noise.
• Figure 5: Compositional evaluation of token embeddings from text and vision tokenization across three languages. Cosine similarity and angle are computed between original full-word embedding (e.g., offline) and its composed embedding (e.g., {off, line}). Sum ("+”) means the composed embedding is obtained by summing subword embeddings. Space ("@”) denotes composition by concatenating subwords with a space. Vision tokenization yields composed embeddings more consistent with the full word across all languages.