Towards Understanding How Knowledge Evolves in Large Vision-Language Models

TL;DR

This work investigates how multimodal knowledge evolves within LVLMs as they transform visual-language inputs into natural language. By tracking next-token probabilities, layerwise token distributions, and high-dimensional feature encodings, the authors identify two pivotal layers—critical layers and mutation layers—that segment knowledge evolution into rapid, stabilized, and mutation phases. They show that token probabilities and distributions exhibit distinct stage-like dynamics and that deep-layer mutations can drive hallucinations, while skip-connection and targeted finetuning strategies offer routes for compression and hallucination mitigation. The findings provide a first principled view of LVLM inner workings and offer practical guidance for efficient, reliable design of multimodal language models.

Abstract

Large Vision-Language Models (LVLMs) are gradually becoming the foundation for many artificial intelligence applications. However, understanding their internal working mechanisms has continued to puzzle researchers, which in turn limits the further enhancement of their capabilities. In this paper, we seek to investigate how multimodal knowledge evolves and eventually induces natural languages in LVLMs. We design a series of novel strategies for analyzing internal knowledge within LVLMs, and delve into the evolution of multimodal knowledge from three levels, including single token probabilities, token probability distributions, and feature encodings. In this process, we identify two key nodes in knowledge evolution: the critical layers and the mutation layers, dividing the evolution process into three stages: rapid evolution, stabilization, and mutation. Our research is the first to reveal the trajectory of knowledge evolution in LVLMs, providing a fresh perspective for understanding their underlying mechanisms. Our codes are available at https://github.com/XIAO4579/Vlm-interpretability.

Figures (7)

• Figure 1: The analysis path in this paper. We begin with the probabilities of single tokens, then delve into the probability distributions of all tokens over the vocabulary set, and finally analyze the hidden features used for generating the tokens.
• Figure 2: The probabilities of tokens across different layers during normal inference processes.
• Figure 3: The probabilities of hallucinated tokens and correct tokens across different layers when the LVLM falls into hallucinations.
• Figure 4: The JS divergences of token probability distributions across adjacent layers during normal inference processes.
• Figure 5: The effect of skip connections on model's output. From left to right: the original image, the descriptions from the original model, the descriptions when skipping from the critical layers to the mutation layers, the descriptions when only skipping the mutation layers, and the descriptions when skipping from the critical layers to the last few layer (as the layers near the output contain linguistic priors, we retain the final 5 layers). Hallucinated tokens are marked in red, and corrected tokens are marked in green.