What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models

TL;DR

This work introduces a two-fold analytical framework featuring a novel probing tool, $\textbf{EmbedLens}$, to conduct a fine-grained analysis of visual token processing, uncovering a pronounced semantic sparsity at the input level.

Abstract

Multimodal large language models (MLLMs) project visual tokens into the embedding space of language models, yet the internal structuring and processing of visual semantics remain poorly understood. In this work, we introduce a two-fold analytical framework featuring a novel probing tool, $\textbf{EmbedLens}$, to conduct a fine-grained analysis. We uncover a pronounced semantic sparsity at the input level: visual tokens consistently partition into sink, dead, and alive categories. Remarkably, only the alive tokens, comprising $\approx60\%$ of the total input, carry image-specific meaning. Furthermore, using a targeted patch-compression benchmark, we demonstrate that these alive tokens already encode rich, fine-grained cues (e.g., objects, colors, and OCR) prior to entering the LLM. Internal visual computations (such as visual attention and feed-forward networks) are redundant for most standard tasks. For the small subset of highly vision-centric tasks that actually benefit from internal processing, we reveal that alive tokens naturally align with intermediate LLM layers rather than the initial embedding space, indicating that shallow-layer processing is unnecessary and that direct mid-layer injection is both sufficient. Ultimately, our findings provide a unified mechanistic view of visual token processing, paving the way for more efficient and interpretable MLLM architectures through selective token pruning, minimized visual computation, and mid-layer injection. The code is released at: https://github.com/EIT-NLP/EmbedLens.

Figures (16)

• Figure 1: Cluster dominance and Cross-image stability in post-projection visual embeddings. (Left) Cluster similarity cross images. (Right) Number of tokens in the top-5 clusters. The cluster index is ranked by the number of tokens.
• Figure 2: (Left) The t-SNE visualization of visual and textual clusters in word embedding space. (Right) An example illustrating the multi-semantic properties of a token within the green-marked text-proximity cluster shown in the left figure.
• Figure 3: Comparison between EmbedLens and LogitLens in label–token matching accuracy. EmbedLens achieves higher semantic recall at shallow and middle layers across multiple MLLMs, suggesting that visual embeddings already encode object-level semantics prior to LLM fusion.
• Figure 4: Formation (Main) and persistence (Insets) of LLM sink tokens. (Left) Cosine similarity between $\langle\mathrm{bos}\rangle$, ViT sinks, and LLM sinks across sublayers. (Right) Layer-wise tracking of the top-5 most similar tokens to $\langle\mathrm{bos}\rangle$.
• Figure 5: Dead tokens exhibit minimal representation change and receive limited cross-modal attention.Left: Layer-wise cosine similarity within each cluster, showing that dead tokens remain highly self-consistent while other tokens evolve substantially. Right: Text-to-visual attention distribution across clusters. Inset: Token-averaged attention comparison.