CroBIM-V: Memory-Quality Controlled Remote Sensing Referring Video Object Segmentation

TL;DR

This work tackles remote-sensing video referring object segmentation under strict online causality by introducing RS-RVOS Bench, a large-scale, causality-aware dataset, and MQC-SAM, a memory-quality controlled online framework. The method combines Temporal Motion Consistency Calibration to refine initial memory with motion priors and a Decoupled Attention Memory Integration mechanism that fuses semantic anchors, short-term evolution, and discriminative prototypes to curb error propagation. Empirical results on RS-RVOS Bench show state-of-the-art performance across region and contour metrics, with ablations confirming the complementary benefits of TMCC and DAMI. The framework advances practical RS-RVOS by enabling robust, memory-driven segmentation in challenging scenes with weak saliency and dynamic occlusions, offering a solid foundation for causality-aware remote-sensing video understanding.

Abstract

Remote sensing video referring object segmentation (RS-RVOS) is challenged by weak target saliency and severe visual information truncation in dynamic scenes, making it extremely difficult to maintain discriminative target representations during segmentation. Moreover, progress in this field is hindered by the absence of large-scale dedicated benchmarks, while existing models are often affected by biased initial memory construction that impairs accurate instance localization in complex scenarios, as well as indiscriminate memory accumulation that encodes noise from occlusions or misclassifications, leading to persistent error propagation. This paper advances RS-RVOS research through dual contributions in data and methodology. First, we construct RS-RVOS Bench, the first large-scale benchmark comprising 111 video sequences, about 25,000 frames, and 213,000 temporal referring annotations. Unlike common RVOS benchmarks where many expressions are written with access to the full video context, our dataset adopts a strict causality-aware annotation strategy in which linguistic references are generated solely from the target state in the initial frame. Second, we propose a memory-quality-aware online referring segmentation framework, termed Memory Quality Control with Segment Anything Model (MQC-SAM). MQC-SAM introduces a temporal motion consistency module for initial memory calibration, leveraging short-term motion trajectory priors to correct structural deviations and establish accurate memory anchoring. Furthermore, it incorporates a decoupled attention-based memory integration mechanism with dynamic quality assessment, selectively updating high-confidence semantic features while filtering unreliable information, thereby effectively preventing error accumulation and propagation. Extensive experiments on RS-RVOS Bench demonstrate that MQC-SAM achieves state-of-the-art performance.

Figures (8)

• Figure 1: Key differences between natural-scene RVOS and RS-RVOS. (a) Natural videos benefit from strong temporal action cues. (b) Remote-sensing RVOS relies on initial anchoring and memory propagation under online causality. (c) Errors can accumulate over time, causing drift.
• Figure 2: Representative characteristics of the RS-RVOS Bench dataset. (a) Visual examples illustrating intrinsic challenges in remote sensing videos, including occlusion over time, background interference from visually similar distractors, and weak target saliency under large-scale scenes. (b) Word cloud of referring expressions, showing the distribution of spatial terms and attribute descriptions used for causality-compliant prompt generation. (c) Statistical distribution of video resolutions in RS-RVOS Bench.
• Figure 3: Overview of the RS-RVOS Bench dataset construction pipeline. Stage 1 applies VDS-based filtering to select visually discriminative sequences. Stage 2 generates causality-compliant referring expressions from the initial frame.
• Figure 4: Overview of the proposed MQC-SAM framework for online remote sensing referring video object segmentation. The framework follows a two-stage design. The left part illustrates Stage 1: Initialization and Calibration, where vision--language interaction and temporal motion-consistency calibration are jointly employed to generate a high-quality initial memory, leveraging adaptive time windows and motion--semantic dual verification to refine initial segmentation and suppress noise. The right part depicts Stage 2: Sequential Segmentation, which performs online inference through a Decoupled Attention Memory Integration mechanism, including a fixed cross-modal semantic anchor for identity consistency, a short-term spatiotemporal FIFO window for appearance adaptation, and discriminative prototypes for error correction. These memory components are fused via weighted attention to enable robust target tracking under strict online causality constraints.
• Figure 5: Temporal motion-consistency initial memory calibration module. The module is followed by motion consistency optimization and semantic--motion dual verification to refine the initial mask and generate the calibrated memory $\tilde{R}_{\text{calibrated}}$.