Enhancing Image Matting in Real-World Scenes with Mask-Guided Iterative Refinement

TL;DR

Mask2Alpha tackles the challenging problem of real-world image matting by integrating mask-guided, self-supervised semantic guidance with iterative refinement. It combines a Mask-Guided Image Encoder, a multi-stage Iterate Decoding pipeline, and a Self-Guided Sparse Detail Recovery module to progressively produce high-quality alpha mattes from coarse to high-resolution representations. Key contributions include a region-aware attention mechanism guided by masks, a unidirectional state-transition refinement scheme with confidence-guided sampling, and adaptive sparse detail recovery to maintain efficiency while preserving boundary detail. The approach demonstrates state-of-the-art performance across diverse real-world datasets, with strong generalization and improved instance awareness, offering a practical, efficient solution for matting in complex scenes.

Abstract

Real-world image matting is essential for applications in content creation and augmented reality. However, it remains challenging due to the complex nature of scenes and the scarcity of high-quality datasets. To address these limitations, we introduce Mask2Alpha, an iterative refinement framework designed to enhance semantic comprehension, instance awareness, and fine-detail recovery in image matting. Our framework leverages self-supervised Vision Transformer features as semantic priors, strengthening contextual understanding in complex scenarios. To further improve instance differentiation, we implement a mask-guided feature selection module, enabling precise targeting of objects in multi-instance settings. Additionally, a sparse convolution-based optimization scheme allows Mask2Alpha to recover high-resolution details through progressive refinement,from low-resolution semantic passes to high-resolution sparse reconstructions. Benchmarking across various real-world datasets, Mask2Alpha consistently achieves state-of-the-art results, showcasing its effectiveness in accurate and efficient image matting.

Figures (5)

• Figure 1: MGM-in-the-wildpark2023mgmwild often fail in real-world applications, particularly when handling fine-grained object details and reducing edge errors. We propose Mask2Alpha to address the difficulties of real-world scenarios.
• Figure 2: Iterative Optimization Process. The Mask2Alpha framework operates in two stages: (a) Semantic Iterative Optimization - begins by refining high-confidence regions through a state transition matrix, where the first row represents the input mask, the second row displays the state transition, and the third row shows the resulting semantic output; (b) Detail Iterative Optimization - progressively enhances uncertain fine details following semantic refinement, aiming to recover the optimal solution across varying resolutions.
• Figure 3: The Pipeline of our Mask2Alpha.The process begins with the Input Image and Initial Mask, which are processed by the Mask-Guided Image Encoder to extract multi-scale features guided by semantic regions. These features are then passed to the Iterative Decoding stage, where alpha mattes are progressively refined over multiple iterations. Finally, the Self-Guided Sparse Detail Recovery stage uses adaptive fusion with confidence-weighted feature maps to output the final refined alpha matte with enhanced high-resolution detail and precision.
• Figure 4: Qualitative Comparisons Across Diverse Real-World Datasets. Our method demonstrates superior generalization ability across various category-diverse real-world datasets, surpassing category-specific models. It shows enhanced semantic understanding, and improved detail-handling capability in complex scenes compared to mask-guided methods.
• Figure 5: Qualitaive results of sparse activation maps. The second row presents sparse activation maps, comparing our method and Sparsematsun2023sparsemat. Our self-guided approach automatically activates more regions based on fine-grained details.