Contribution-aware Token Compression for Efficient Video Understanding via Reinforcement Learning

TL;DR

CaCoVID tackles the inefficiency of dense video tokens in video LLMs by learning a contribution-aware compression policy through reinforcement learning. It introduces online combinatorial space sampling (OCSS) to efficiently explore token subsets, reducing the effective search space from $2^{n}$ to structured subspaces. Empirically, CaCoVID achieves state-of-the-art Avg. Acc across multiple retention ratios with substantially lower latency than existing content-based and model-based compression methods. The approach enables practical, query-aware video understanding on large-scale LLMs without requiring re-training of the underlying LLM.

Abstract

Video large language models have demonstrated remarkable capabilities in video understanding tasks. However, the redundancy of video tokens introduces significant computational overhead during inference, limiting their practical deployment. Many compression algorithms are proposed to prioritize retaining features with the highest attention scores to minimize perturbations in attention computations. However, the correlation between attention scores and their actual contribution to correct answers remains ambiguous. To address the above limitation, we propose a novel \textbf{C}ontribution-\textbf{a}ware token \textbf{Co}mpression algorithm for \textbf{VID}eo understanding (\textbf{CaCoVID}) that explicitly optimizes the token selection policy based on the contribution of tokens to correct predictions. First, we introduce a reinforcement learning-based framework that optimizes a policy network to select video token combinations with the greatest contribution to correct predictions. This paradigm shifts the focus from passive token preservation to active discovery of optimal compressed token combinations. Secondly, we propose a combinatorial policy optimization algorithm with online combination space sampling, which dramatically reduces the exploration space for video token combinations and accelerates the convergence speed of policy optimization. Extensive experiments on diverse video understanding benchmarks demonstrate the effectiveness of CaCoVID. Codes will be released.

Figures (8)

• Figure 1: As observed, the attention scores of LLMs demonstrate ambiguity in their correlation with token contributions to correct question answering. Higher attention scores are not allocated to critical tokens such as "the clothing of the man", which may stem from the visual attention sink phenomenon attentionsink. In contrast, our contribution scores learned through LLM prediction feedback effectively focus on the regions most critical to answering the question correctly - specifically, the "the clothing of the man" area.
• Figure 2: (a) illustrates the training pipeline of contribution-aware token compression for video understanding (CaCoVID), which optimizes the compression policy network by exploring the contributions of different token combinations to correct predictions. (b) presents the architecture of the compression policy network, which builds interactions between video tokens and question tokens in a self-attention mechanism and estimates the contributions of video tokens or frames to the correct prediction by MLPs.
• Figure 3: Illustration of the online combinatorial space sampling strategy. Tokens are first sorted by their contribution scores estimated by the policy network and partitioned into combinatorial sub-spaces. Then, a categorical distribution is applied to sample the subspace according to the sum of contribution scores within each subspace. Finally, tokens within the selected subspace are sampled according to a multinomial distribution to acquire the final token combination.
• Figure 4: Our OCSS can reduce the logarithmic exploration space complexity ${\rm log(\mathcal{C})}$ to 1/25 of arbitrary exploration magnitude.
• Figure 5: The compression policy network can effectively identify the most critical frames to answer the question, such as the frames when the man picks up a yellow cloth and the frames when the girl looks into the computer at the end of the video.