Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PP-Matting: High-Accuracy Natural Image Matting

Guowei Chen, Yi Liu, Jian Wang, Juncai Peng, Yuying Hao, Lutao Chu, Shiyu Tang, Zewu Wu, Zeyu Chen, Zhiliang Yu, Yuning Du, Qingqing Dang, Xiaoguang Hu, Dianhai Yu

TL;DR

Natural image matting is ill-posed and often requires user-provided trimaps, limiting real-world use. PP-Matting proposes a trimap-free architecture with a shared encoder and two parallel branches: a Semantic Context Branch for global semantics and a High-Resolution Detail Branch for fine details, connected by a guidance flow to produce a semantic-aware alpha matte. It trains with three losses—semantic, detail, and fusion—forming a weighted combination that enforces correct semantics and sharp transitions. The approach achieves strong results on Composition-1k and Distinctions-646 and demonstrates practical human matting capabilities, with code to be released for reproducibility.

Abstract

Natural image matting is a fundamental and challenging computer vision task. It has many applications in image editing and composition. Recently, deep learning-based approaches have achieved great improvements in image matting. However, most of them require a user-supplied trimap as an auxiliary input, which limits the matting applications in the real world. Although some trimap-free approaches have been proposed, the matting quality is still unsatisfactory compared to trimap-based ones. Without the trimap guidance, the matting models suffer from foreground-background ambiguity easily, and also generate blurry details in the transition area. In this work, we propose PP-Matting, a trimap-free architecture that can achieve high-accuracy natural image matting. Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground with keeping feature resolution unchanged. Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask. It prevents the detail prediction from local ambiguity caused by semantic context missing. In addition, we conduct extensive experiments on two well-known benchmarks: Composition-1k and Distinctions-646. The results demonstrate the superiority of PP-Matting over previous methods. Furthermore, we provide a qualitative evaluation of our method on human matting which shows its outstanding performance in the practical application. The code and pre-trained models will be available at PaddleSeg: https://github.com/PaddlePaddle/PaddleSeg.

PP-Matting: High-Accuracy Natural Image Matting

TL;DR

Natural image matting is ill-posed and often requires user-provided trimaps, limiting real-world use. PP-Matting proposes a trimap-free architecture with a shared encoder and two parallel branches: a Semantic Context Branch for global semantics and a High-Resolution Detail Branch for fine details, connected by a guidance flow to produce a semantic-aware alpha matte. It trains with three losses—semantic, detail, and fusion—forming a weighted combination that enforces correct semantics and sharp transitions. The approach achieves strong results on Composition-1k and Distinctions-646 and demonstrates practical human matting capabilities, with code to be released for reproducibility.

Abstract

Natural image matting is a fundamental and challenging computer vision task. It has many applications in image editing and composition. Recently, deep learning-based approaches have achieved great improvements in image matting. However, most of them require a user-supplied trimap as an auxiliary input, which limits the matting applications in the real world. Although some trimap-free approaches have been proposed, the matting quality is still unsatisfactory compared to trimap-based ones. Without the trimap guidance, the matting models suffer from foreground-background ambiguity easily, and also generate blurry details in the transition area. In this work, we propose PP-Matting, a trimap-free architecture that can achieve high-accuracy natural image matting. Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground with keeping feature resolution unchanged. Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask. It prevents the detail prediction from local ambiguity caused by semantic context missing. In addition, we conduct extensive experiments on two well-known benchmarks: Composition-1k and Distinctions-646. The results demonstrate the superiority of PP-Matting over previous methods. Furthermore, we provide a qualitative evaluation of our method on human matting which shows its outstanding performance in the practical application. The code and pre-trained models will be available at PaddleSeg: https://github.com/PaddlePaddle/PaddleSeg.
Paper Structure (12 sections, 12 equations, 6 figures, 4 tables)

This paper contains 12 sections, 12 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Approach
  4. Network Architecture
  5. Loss Function
  6. Experiments
  7. Datasets
  8. Implementation Details
  9. Evaluation Results
  10. Ablation Study
  11. Human Matting
  12. Conclusion

Figures (6)

  • Figure 1: Comparison of segmentation and matting. The original image is on the left, the segmentation results is in the middle, and the matting results is on the right.
  • Figure 2: The overview of network architecture. HRNet is used as the encoder, the low-resolution output is used as the input of PPM, and the high-resolution output is used as the input of the high-resolution detail branch (HRDB). The semantic context branch (SCB) has five blocks, and the first, third, and fifth blocks guide the HRDB to learn semantic context. The HRDB maintains high-resolution inference to have a high-quality detail prediction.
  • Figure 3: Guided flow architecture
  • Figure 4: Visualization of the features and results. Alpha matte is the final prediction result. s1, s2, and s3 are the feature maps from the SCB. Low-level feature is from the first block of the encoder. g1, g2, and g3 are the output of GCLs. Detail map is the output of HRDB.
  • Figure 5: Example of the real human image. The real image is from AIM-500li2021deep.
  • ...and 1 more figures