SCoPE VLM: Selective Context Processing for Efficient Document Navigation in Vision-Language Models

TL;DR

SCoPE VLM addresses long-context multimodal document understanding by introducing Chain of Scroll (CoS), an action-based navigation mechanism that selectively visits document segments. The approach is supported by the SCoPE Dataset for supervised and reinforcement-learning training and by Episodic Group Relative Policy Optimization (EGRPO), which optimizes penultimate-to-terminal decisions and terminal outcomes under memory constraints. Empirical results show SCoPE VLM achieves competitive accuracy with substantially lower memory usage than baseline long-context methods and demonstrates transferable document navigation skills to GUI control tasks. This work advances locally deployable, agentic multimodal systems capable of efficient, human-like reading and decision-making in long documents and structured GUI/web contexts.

Abstract

Understanding long-context visual information remains a fundamental challenge for vision-language models, particularly in agentic tasks such as GUI control and web navigation. While web pages and GUI environments are inherently structured documents, current VLMs typically neglect decision-oriented document understanding in their training objectives. Existing approaches primarily extend visual embeddings to process long, high-resolution inputs, but these methods are memory-intensive and impractical for locally deployable solutions. To address these issues, we propose SCoPE VLM, a document navigation expert that leverages a novel Chain of Scroll mechanism to selectively and recursively navigate documents, focusing exclusively on relevant segments. We introduce a dedicated data generation pipeline to construct informative Chain of Scroll trajectories and Episodic Group Relative Policy Optimization, a tailored reinforcement learning method to reduce the gap between training and inference. Our method substantially reduces memory usage and effectively models human-like reading behaviors. To the best of our knowledge, SCoPE VLM is the first framework to explicitly model agentic reading patterns in multi-page document question answering, advancing the capabilities of multimodal agents.

Figures (18)

• Figure 1: Performance-efficiency comparison of SCoPE VLM variants against Qwen-2.5-VL series on M3DocVQA and SlideVQA from Table \ref{['tab:vram']}, Section \ref{['sec:Experiments']}, achieving 2.38x higher efficiency than the baseline.
• Figure 2: Chain-of-Scroll navigation trajectories. SCoPE VLM with EGRPO efficiently locates telephone information in a 19-page document, visiting only 9 pages.
• Figure 3: Overview of the Chain-of-Scroll framework. The model iteratively decides whether to scroll to new pages or generate an answer based on accumulated context and relevance signals, emulating human-like selective document navigation.
• Figure 4: Overview of Episodic Group Relative Policy Optimization. The tailored framework for CoS employs group-based sampling with top-N selection during document navigation. Terminal step projection from the penultimate state enables return estimation that guides the model toward correct answers.
• Figure 5: Input prompt of CoS framework. At every step, the blue part is replaced with the input query, accumulated previous notes, current page number, and total page number until the model chooses to return the answer.