Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation

Xusheng He, Canyang Wu, Jinrong Zhang, Weili Guan, Jianlong Wu, Liqiang Nie

Abstract

This report presents our winning solution to the 5th PVUW MeViS-Text Challenge. The track studies referring video object segmentation under motion-centric language expressions, where the model must jointly understand appearance, temporal behavior, and object interactions. To address this problem, we build a fully training-free pipeline that combines strong multimodal large language models with SAM3. Our method contains three stages. First, Gemini-3.1 Pro decomposes each target event into instance-level grounding targets, selects the frame where the target is most clearly visible, and generates a discriminative description. Second, SAM3-agent produces a precise seed mask on the selected frame, and the official SAM3 tracker propagates the mask through the whole video. Third, a refinement stage uses Qwen3.5-Plus and behavior-level verification to correct ambiguous or semantically inconsistent predictions. Without task-specific fine-tuning, our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897. The code is available at https://github.com/Moujuruo/MeViSv2_Track_Solution_2026.

The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation

Abstract

This report presents our winning solution to the 5th PVUW MeViS-Text Challenge. The track studies referring video object segmentation under motion-centric language expressions, where the model must jointly understand appearance, temporal behavior, and object interactions. To address this problem, we build a fully training-free pipeline that combines strong multimodal large language models with SAM3. Our method contains three stages. First, Gemini-3.1 Pro decomposes each target event into instance-level grounding targets, selects the frame where the target is most clearly visible, and generates a discriminative description. Second, SAM3-agent produces a precise seed mask on the selected frame, and the official SAM3 tracker propagates the mask through the whole video. Third, a refinement stage uses Qwen3.5-Plus and behavior-level verification to correct ambiguous or semantically inconsistent predictions. Without task-specific fine-tuning, our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897. The code is available at https://github.com/Moujuruo/MeViSv2_Track_Solution_2026.

Paper Structure

This paper contains 9 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Our solution
  3. Framework Overview
  4. Three-Stage Pipeline
  5. Experiments
  6. Challenge Description
  7. Implementation Details
  8. Results
  9. Conclusion

Figures (2)

  • Figure 1: Overview of our training-free pipeline for the PVUW MeViS-Text challenge. Stage 1 uses Gemini-3.1 Pro to decompose each target event into instance-level grounding targets, select the clearest frame, and generate a discriminative description. Stage 2 uses SAM3-agent to generate seed masks on the selected frames and propagates them through the video with the SAM3 tracker. Stage 3 further refines ambiguous or semantically inconsistent predictions through mask inspection, description regeneration, and behavior-level verification.
  • Figure 2: Qualitative visualization of our method on the MeViS-Text track. The figure shows representative frames together with the predicted masks, as well as an example of discriminative description generation and SAM3-agent grounding.