Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs

TL;DR

VideoLLMs extend vision-language models to process spatiotemporal input for VideoQA, but little is known about how they extract and propagate temporal information. The authors apply mechanistic interpretability, notably Attention Knockout and Logit Lens analyses, to map information flow across layers and modalities, uncovering a four-stage pattern of temporal reasoning: cross-frame video interactions in early-to-middle layers, then alignment with temporal keyword concepts in middle layers, convergence onto the last token in middle-to-late layers, and answer generation thereafter. They demonstrate that a small set of effective information pathways suffices for solving VideoQA, as disabling non-critical connections leaves performance largely intact across multiple VideoLLMs and tasks. These findings offer a concrete blueprint for understanding VideoLLMs’ temporal reasoning and provide practical guidance to improve interpretability and generalization in video-based multimodal models.

Abstract

Video Large Language Models (VideoLLMs) extend the capabilities of vision-language models to spatiotemporal inputs, enabling tasks such as video question answering (VideoQA). Despite recent advances in VideoLLMs, their internal mechanisms on where and how they extract and propagate video and textual information remain less explored. In this study, we investigate the internal information flow of VideoLLMs using mechanistic interpretability techniques. Our analysis reveals consistent patterns across diverse VideoQA tasks: (1) temporal reasoning in VideoLLMs initiates with active cross-frame interactions in early-to-middle layers, (2) followed by progressive video-language integration in middle layers. This is facilitated by alignment between video representations and linguistic embeddings containing temporal concepts. (3) Upon completion of this integration, the model is ready to generate correct answers in middle-to-late layers. (4) Based on our analysis, we show that VideoLLMs can retain their VideoQA performance by selecting these effective information pathways while suppressing a substantial amount of attention edges, e.g., 58% in LLaVA-NeXT-7B-Video-FT. These findings provide a blueprint on how VideoLLMs perform temporal reasoning and offer practical insights for improving model interpretability and downstream generalization. Our project page with the source code is available at https://map-the-flow.github.io

Figures (27)

• Figure 0: Change in prediction probability when disconnecting cross-frame attention edges in LLaVA-NeXT-13B-Video-FT and LLaVA-NeXT-13B.
• Figure 1: Summary of our findings on VideoLLMs' information flow.(a) Temporal reasoning begins with cross-frame interactions within video tokens at early-middle layers [green], followed by video-language integration into temporal keywords in the question [purple]. This information is conveyed to the last token at middle-late layers [orange], where answer generation occurs [yellow]. (b) These effective pathways are identified via Attention Knockout, which disconnects attention pairs and tracks the drop in probability of the final answer to quantify their impact. (c) Layer-wise answer probability rises immediately after video-language integration, indicating that the model is ready to predict correct answers after the middle layers.
• Figure 2: Change in prediction probability when disconnecting cross-frame attention edges. Blocking cross-frame interactions in early-to-middle layers significantly harms LLaVA-NeXT-7B-Video-FT’s prediction, while LLaVA-NeXT-7B remains mostly unaffected.
• Figure 3: Overall cross-modal information flow in VideoLLMs. We analyze changes in the prediction probability when intervening on attention edges between video, question, and last token (i.e., the starting position for answer generation), following the protocol of zhang2024crossmodal. Information from the video tokens is conveyed to the question tokens in the early-to-middle layers, followed by the transfer of information from the question tokens to the last token in the middle-to-late layers. Source$\nrightarrow$Target indicates blocking attention edges from source tokens to the target tokens.
• Figure 4: Normalized frequency of spatial and temporal keywords extracted from video tokens via Logit Lens. Spatial concepts start to appear in the very early layers, whereas temporal concepts develop later in the middle layers. Full list of keywords are shown in Table \ref{['tab:vocabulary_pool']}.