LongStream: Long-Sequence Streaming Autoregressive Visual Geometry

TL;DR

This work introduces LongStream, a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction across thousands of frames, which suppresses attention degradation over ultra-long sequences and reduces the gap between training and inference.

Abstract

Long-sequence streaming 3D reconstruction remains a significant open challenge. Existing autoregressive models often fail when processing long sequences. They typically anchor poses to the first frame, which leads to attention decay, scale drift, and extrapolation errors. We introduce LongStream, a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction across thousands of frames. Our approach is threefold. First, we discard the first-frame anchor and predict keyframe-relative poses. This reformulates long-range extrapolation into a constant-difficulty local task. Second, we introduce orthogonal scale learning. This method fully disentangles geometry from scale estimation to suppress drift. Finally, we solve Transformer cache issues such as attention-sink reliance and long-term KV-cache contamination. We propose cache-consistent training combined with periodic cache refresh. This approach suppresses attention degradation over ultra-long sequences and reduces the gap between training and inference. Experiments show LongStream achieves state-of-the-art performance. It delivers stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS. Project Page: https://3dagentworld.github.io/longstream/

Figures (8)

• Figure 1: Streaming Autoregressive Model Comparison for Metric-Scale Scene Reconstruction. Existing streaming models (e.g., Stream3R lan2025stream3r, StreamVGGT zhuo2025streaming) collapse within tens of meters, suffering from extrapolation errors. In contrast, our proposed LongStream delivers stable, kilometer-scale reconstruction. Its gauge-decoupled formulation and cache-consistent inference preserve metric accuracy and geometric stability, sustaining 18 FPS performance across multi-kilometer sequences.
• Figure 2: Memory and runtime comparison. Our method keeps memory and latency stable, whereas VGGT and FastVGGT grow rapidly and hit OOM on long sequences.
• Figure 3: Overview of our proposed LongStream. Given streaming inputs, patch tokens are extracted by a ViT encoder and augmented with keyframe, normal-frame, and scale tokens. Tokens are fused via causal attention with a shared KV cache, which is consistently used in both training and inference for cache-consistent streaming modeling. The network predicts keyframe-relative poses $\mathbf{T}_{i\leftarrow k}$, depth, pointmap, and global scale, enabling stable metric-scale reconstruction over long sequences.
• Figure 4: Cache-consistent training (CCT). We show attention maps (top) and Relative Pose Error (RPE) heatmaps (bottom) under different training–inference settings. Without CCT (left), causal inference develops a strong attention sink; windowed inference either amplifies this sink when it is kept or collapses when it is removed. With CCT (right), the sink is strongly suppressed in causal mode and likewise suppressed in both windowed modes, yielding stable and best accuracy. Light blue denotes attention to the keyframe.
• Figure 5: Qualitative comparison on long-sequence pose estimation. We compare LongStream against streaming and SLAM baselines on KITTI and vKITTI sequences spanning several hundred meters. Stream3R and StreamVGGT accumulate drift over long trajectories, and VGGT-SLAM runs out of memory on the second vKITTI sequence. LongStream preserves stable and coherent poses across all scenes, maintaining trajectory continuity even under large loop motions.