EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial Intelligence

TL;DR

EagleVision introduces a dual-stage spatial reasoning framework that separates macro perception from micro verification to tackle core challenges in spatial CoT, including global viewpoint coverage under token budgets and verifiable, BEV-grounded evidence. The macro stage uses SPF-DPP to select a compact, geometry-aware frame set by jointly optimizing semantic relevance and SE(3) viewpoint diversity, while the micro stage performs BEV-grounded chain-of-thought using pose querying and RL with a spatial grounding reward. The approach yields state-of-the-art results among open-source vision-language models on VSI-Bench and demonstrates strong generalization through ablations and qualitative analyses. This work advances traceable, geometry-consistent spatial reasoning for embodied perception tasks and sets the stage for more unified, end-to-end spatial reasoning systems.

Abstract

Recent spatial intelligence approaches typically attach 3D cues to 2D reasoning pipelines or couple MLLMs with black-box reconstruction modules, leading to weak spatial consistency, limited viewpoint diversity, and evidence chains that cannot be traced back to supporting views. Frameworks for "thinking with images" (e.g., ChatGPT-o3 and DeepEyes) show that stepwise multimodal reasoning can emerge by interleaving hypothesis formation with active acquisition of visual evidence, but they do not address three key challenges in spatial Chain-of-Thought (CoT): building global space perception under strict token budgets, explicitly associating 3D hypotheses with video frames for verification, and designing spatially grounded rewards for reinforcement learning. To address these issues, we present EagleVision, a dual-stage framework for progressive spatial cognition through macro perception and micro verification. In the macro perception stage, EagleVision employs a semantics-perspective-fusion determinantal point process (SPF-DPP) to select a compact set of geometry- and semantics-aware keyframes from long videos under a fixed token budget. In the micro verification stage, we formalize spatial CoT as BEV-grounded pose querying: the agent iteratively predicts poses on a BEV plane, retrieves the nearest real frames, and is trained purely by reinforcement learning with a spatial grounding reward that scores the consistency between predicted poses and observed views. On VSI-Bench, EagleVision achieves state-of-the-art performance among open-source vision-language models, demonstrating strong and generalizable spatial understanding.

Figures (7)

• Figure 1: Overview of EagleVision. Compared with conventional instruction-following or text-only Chain-of-Thought (CoT) models that lack explicit spatial reasoning, EagleVision is a dual-stage framework for progressive spatial cognition through macro perception and micro verification. (1) Macro perception selects a compact yet spatially informative set of frames by jointly considering semantic relevance and geometric diversity under a token budget. (2) Micro verification formalize spatial multi-modal CoT as BEV-grounding based pose querying with a spatial reward for verifiable spatial observations, forming a verifiable evidence chain for interpretable, geometry-aware answers.
• Figure 2: Framework of EagleVision. The framework consists of two stages for progressive spatial cognition: (i) Macro perception for selecting spatially informative frames under a token budget, and (ii) Micro verification with spatial thinking to refine spatial understanding.
• Figure 3: Macro Perception. Given an input video and a question, we first reconstruct the scene geometry and camera trajectory in $\mathrm{SE}(3)$ to build a sparse pose graph, whose heat-kernel diffusion yields the viewpoint kernel $K_{\mathrm{view}}$. In parallel, FG-CLIP xie2025fg computes frame–keyword similarities that are temperature-calibrated into semantic scores and encoded in a diagonal quality matrix $Q$. The resulting DPP L-ensemble $L_{\mathrm{dpp}} = Q K_{\mathrm{view}} Q$ is then used to greedily select $k$ frames.
• Figure 4: Spatial MCoT for Micro Verification. The model begins with videos, questions and BEV image, then queries on the BEV plane, retrieves the nearest video frame as an activity observation, encodes it using a visual encoder, and feeds it into a large language model to generate interleaved spatial thought chains for iterative spatial reasoning and thinking.
• Figure 5: Visualization of SPF-DPP of Scannet Scene$00\_00$. (a) The matrix of Sparse Viewpoint Graph and Diffusion Viewpoint Kernel. The smaller sub-figure serves as an enhancement. (b) The Semantic Score and related video frame. The keyword of question is chair, bed and sofa, as similar as the Fig. \ref{['fig:teaser']}.