Spatial-Conditioned Reasoning in Long-Egocentric Videos
James Tribble, Hao Wang, Si-En Hong, Chaoyi Zhou, Ashish Bastola, Siyu Huang, Abolfazl Razi
TL;DR
This work investigates how explicit spatial signals impact vision-language model reasoning on long-horizon egocentric video without altering model architectures. By introducing Sanpo-D, a fine-grained spatial QA benchmark built on SANPO and evaluating multiple VLMs, the study reveals a trade-off between general-purpose accuracy and spatial specialization. The depth-fusion experiments show that incorporating depth priors can improve spatial understanding in safety-critical tasks like obstruction and pedestrian detection, though effects are model-dependent. The findings highlight the potential and limits of input-level spatial cues for open-ended navigation, pointing to future directions in motion cues, temporal geometry, and targeted fine-tuning to improve spatial consistency over time.
Abstract
Long-horizon egocentric video presents significant challenges for visual navigation due to viewpoint drift and the absence of persistent geometric context. Although recent vision-language models perform well on image and short-video reasoning, their spatial reasoning capability in long egocentric sequences remains limited. In this work, we study how explicit spatial signals influence VLM-based video understanding without modifying model architectures or inference procedures. We introduce Sanpo-D, a fine-grained re-annotation of the Google Sanpo dataset, and benchmark multiple VLMs on navigation-oriented spatial queries. To examine input-level inductive bias, we further fuse depth maps with RGB frames and evaluate their impact on spatial reasoning. Our results reveal a trade-off between general-purpose accuracy and spatial specialization, showing that depth-aware and spatially grounded representations can improve performance on safety-critical tasks such as pedestrian and obstruction detection.