Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th LSVOS

TL;DR

The paper tackles video object segmentation in complex and long-term scenarios by introducing a discriminative spatial-temporal VOS framework that combines a spatial-semantic block with discriminative query generation. The spatial-semantic block fuses Vision Transformer-derived semantic priors with multi-scale features and local spatial fusion, while the discriminative query generation refine target queries using the most distinctive features to maintain stable memory during long videos. Training on a large MEGA+LaSOT corpus and evaluating on MOSE and LVOS, the approach achieves a top result of $80.90\%$ $\mathcal{J}\&\mathcal{F}$ on the 6th LSVOS VOS track, demonstrating robust performance against occlusion, reappearance, and small-target challenges. This work advances long-term VOS by integrating semantic awareness and discriminative memory updates, offering practical benefits for real-world video analysis and interactive editing, with code to be released.

Abstract

Video object segmentation (VOS) is a crucial task in computer vision, but current VOS methods struggle with complex scenes and prolonged object motions. To address these challenges, the MOSE dataset aims to enhance object recognition and differentiation in complex environments, while the LVOS dataset focuses on segmenting objects exhibiting long-term, intricate movements. This report introduces a discriminative spatial-temporal VOS model that utilizes discriminative object features as query representations. The semantic understanding of spatial-semantic modules enables it to recognize object parts, while salient features highlight more distinctive object characteristics. Our model, trained on extensive VOS datasets, achieved first place (\textbf{80.90\%} $\mathcal{J \& F}$) on the test set of the 6th LSVOS challenge in the VOS Track, demonstrating its effectiveness in tackling the aforementioned challenges. The code will be available at \href{https://github.com/yahooo-m/VOS-Solution}{code}.

Figures (2)

• Figure 1: Overall framework of our methods.
• Figure 2: Qualitative results on LSVOS sequences.