Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Unsupervised Video Object Segmentation via Fake Flow Generation

Suhwan Cho, Minhyeok Lee, Jungho Lee, Donghyeong Kim, Seunghoon Lee, Sungmin Woo, Sangyoun Lee

TL;DR

This work tackles data scarcity in unsupervised video object segmentation by introducing Fake Flow Generation, a method that synthesizes optical-flow-like inputs from single RGB images using depth estimates. By converting depth maps into plausible two-channel motion signals and augmenting them, the authors create large-scale image-flow pairs to train a simple two-stream encoder–decoder network. The proposed two-stage training protocol—pre-training on YouTube-VOS 2018 and fine-tuning on DAVIS 2016 and DUTSv2—yields state-of-the-art performance on multiple benchmarks, even without complex modules or post-processing. The approach highlights the power of data generation and suggests a robust, scalable baseline for future VOS research, with potential to reduce reliance on real flows and video-labeled data.

Abstract

Unsupervised video object segmentation (VOS), also known as video salient object detection, aims to detect the most prominent object in a video at the pixel level. Recently, two-stream approaches that leverage both RGB images and optical flow maps have gained significant attention. However, the limited amount of training data remains a substantial challenge. In this study, we propose a novel data generation method that simulates fake optical flows from single images, thereby creating large-scale training data for stable network learning. Inspired by the observation that optical flow maps are highly dependent on depth maps, we generate fake optical flows by refining and augmenting the estimated depth maps of each image. By incorporating our simulated image-flow pairs, we achieve new state-of-the-art performance on all public benchmark datasets without relying on complex modules. We believe that our data generation method represents a potential breakthrough for future VOS research.

Improving Unsupervised Video Object Segmentation via Fake Flow Generation

TL;DR

This work tackles data scarcity in unsupervised video object segmentation by introducing Fake Flow Generation, a method that synthesizes optical-flow-like inputs from single RGB images using depth estimates. By converting depth maps into plausible two-channel motion signals and augmenting them, the authors create large-scale image-flow pairs to train a simple two-stream encoder–decoder network. The proposed two-stage training protocol—pre-training on YouTube-VOS 2018 and fine-tuning on DAVIS 2016 and DUTSv2—yields state-of-the-art performance on multiple benchmarks, even without complex modules or post-processing. The approach highlights the power of data generation and suggests a robust, scalable baseline for future VOS research, with potential to reduce reliance on real flows and video-labeled data.

Abstract

Unsupervised video object segmentation (VOS), also known as video salient object detection, aims to detect the most prominent object in a video at the pixel level. Recently, two-stream approaches that leverage both RGB images and optical flow maps have gained significant attention. However, the limited amount of training data remains a substantial challenge. In this study, we propose a novel data generation method that simulates fake optical flows from single images, thereby creating large-scale training data for stable network learning. Inspired by the observation that optical flow maps are highly dependent on depth maps, we generate fake optical flows by refining and augmenting the estimated depth maps of each image. By incorporating our simulated image-flow pairs, we achieve new state-of-the-art performance on all public benchmark datasets without relying on complex modules. We believe that our data generation method represents a potential breakthrough for future VOS research.
Paper Structure (11 sections, 7 equations, 7 figures, 7 tables)

This paper contains 11 sections, 7 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. Preliminaries
  5. Fake Flow Generation
  6. Implementation Details
  7. Experiments
  8. Quantitative Results
  9. Qualitative Results
  10. Analysis
  11. Conclusion

Figures (7)

  • Figure 1: Visualization of our proposed training data construction strategy. By simulating optical flow maps from a large number of single images, large-scale image-flow pairs can be used for network training.
  • Figure 2: Overall pipeline of our fake flow generation process.
  • Figure 3: Visualized depth-to-flow conversion process. Augmented motion indicates the motion map after applying random value shifting and scaling.
  • Figure 4: Example image-flow pairs from DUTSv2 dataset.
  • Figure 5: Qualitative comparison between state-of-the-art methods and our proposed FakeFlow.
  • ...and 2 more figures