Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Real-World High-Precision Image Matting and Segmentation

Haipeng Zhou, Zhaohu Xing, Hongqiu Wang, Jun Ma, Ping Li, Lei Zhu

TL;DR

This work tackles real-world high-precision image matting and dichotomous segmentation by addressing domain gaps between synthetic and real data and enabling interactive, target-specific predictions. The proposed Foreground Consistent Learning Model (FCLM) combines depth-aware distillation from a strong teacher to a compact student, foreground-consistent domain adaptation with adversarial and optimal-transport alignment, and an Object-Oriented Decoder that accepts both visual and language prompts. Key contributions include treating synthetic data as a domain adaptation problem, introducing depth-guided feature distillation, a foreground-centric alignment strategy, and a prompt-ready decoder architecture; together these yield state-of-the-art results on HIM2K, RefMatte, and DIS-5K. The framework supports flexible user interaction and semantic guidance, offering practical benefits for AR/VR, image editing, and open-world segmentation tasks with robust real-world generalization.

Abstract

High-precision scene parsing tasks, including image matting and dichotomous segmentation, aim to accurately predict masks with extremely fine details (such as hair). Most existing methods focus on salient, single foreground objects. While interactive methods allow for target adjustment, their class-agnostic design restricts generalization across different categories. Furthermore, the scarcity of high-quality annotation has led to a reliance on inharmonious synthetic data, resulting in poor generalization to real-world scenarios. To this end, we propose a Foreground Consistent Learning model, dubbed as FCLM, to address the aforementioned issues. Specifically, we first introduce a Depth-Aware Distillation strategy where we transfer the depth-related knowledge for better foreground representation. Considering the data dilemma, we term the processing of synthetic data as domain adaptation problem where we propose a domain-invariant learning strategy to focus on foreground learning. To support interactive prediction, we contribute an Object-Oriented Decoder that can receive both visual and language prompts to predict the referring target. Experimental results show that our method quantitatively and qualitatively outperforms SOTA methods.

Toward Real-World High-Precision Image Matting and Segmentation

TL;DR

This work tackles real-world high-precision image matting and dichotomous segmentation by addressing domain gaps between synthetic and real data and enabling interactive, target-specific predictions. The proposed Foreground Consistent Learning Model (FCLM) combines depth-aware distillation from a strong teacher to a compact student, foreground-consistent domain adaptation with adversarial and optimal-transport alignment, and an Object-Oriented Decoder that accepts both visual and language prompts. Key contributions include treating synthetic data as a domain adaptation problem, introducing depth-guided feature distillation, a foreground-centric alignment strategy, and a prompt-ready decoder architecture; together these yield state-of-the-art results on HIM2K, RefMatte, and DIS-5K. The framework supports flexible user interaction and semantic guidance, offering practical benefits for AR/VR, image editing, and open-world segmentation tasks with robust real-world generalization.

Abstract

High-precision scene parsing tasks, including image matting and dichotomous segmentation, aim to accurately predict masks with extremely fine details (such as hair). Most existing methods focus on salient, single foreground objects. While interactive methods allow for target adjustment, their class-agnostic design restricts generalization across different categories. Furthermore, the scarcity of high-quality annotation has led to a reliance on inharmonious synthetic data, resulting in poor generalization to real-world scenarios. To this end, we propose a Foreground Consistent Learning model, dubbed as FCLM, to address the aforementioned issues. Specifically, we first introduce a Depth-Aware Distillation strategy where we transfer the depth-related knowledge for better foreground representation. Considering the data dilemma, we term the processing of synthetic data as domain adaptation problem where we propose a domain-invariant learning strategy to focus on foreground learning. To support interactive prediction, we contribute an Object-Oriented Decoder that can receive both visual and language prompts to predict the referring target. Experimental results show that our method quantitatively and qualitatively outperforms SOTA methods.
Paper Structure (23 sections, 9 equations, 6 figures, 6 tables)

This paper contains 23 sections, 9 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Dense Prediction
  4. High-Precision Image Parsing
  5. Data Reliability
  6. Methodology
  7. Overview
  8. Depth-Aware Distillation
  9. Foreground Consistent Domain Adaptation
  10. Object-Oriented Decoder
  11. Optimization Objective
  12. Experiments
  13. Setups
  14. Datasets and Metrics.
  15. Implement details.
  16. ...and 8 more sections

Figures (6)

  • Figure 1: Overview of the framework of our proposed FCLM. During training, we input two images sharing the same foreground. First, we perform Depth-Aware Distillation to transfer depth-related knowledge from the teacher model to the student model. Then, we apply Foreground Consistent Domain Adaptation to enhance the generalization ability of the model. Our prompt encoder supports various types of prompts, enabling multi-instance and semantic-aware high-precision prediction.
  • Figure 2: Illustration of the Depth-Aware Distillation. We deploy depth map as guidance to drive foreground and background distillation.
  • Figure 3: Illustration of our Object-Oriented Decoder. It supports various prompt inputs to orient the object, achieving interactive high-precision matting or segmentation.
  • Figure 4: Visual comparison with on HIM2K and RefMatte real-world multi-objects datasets. Note that ours is guided by text, while others are used by visual prompt (box).
  • Figure 5: Visual comparisons with different methods on DIS-5K dataset. Zoom in for the best view.
  • ...and 1 more figures