Boosting General Trimap-free Matting in the Real-World Image

TL;DR

This work tackles the limitations of trimap-free matting by redefining the foreground as the salient object and introducing a coarse-to-fine MFC-Net that fuses multi-level features to produce complete alpha mattes. A novel data-harmony strategy bridges the gap between synthetic and real-world images, and a large Real-19k dataset provides robust real-world evaluation. The model combines a coarse module with Head Attention, Global Context Flow, and Inter-weaved Aggregation to estimate a foreground, followed by a refine module that produces a high-quality alpha matte with edge-aware fusion. Experiments show state-of-the-art results among trimap-free methods on Real-19k and competitive performance on synthetic datasets, with strong generalization to varied real-world scenes, enabled by image harmony and the large real-world dataset.

Abstract

Image matting aims to obtain an alpha matte that separates foreground objects from the background accurately. Recently, trimap-free matting has been well studied because it requires only the original image without any extra input. Such methods usually extract a rough foreground by itself to take place trimap as further guidance. However, the definition of 'foreground' lacks a unified standard and thus ambiguities arise. Besides, the extracted foreground is sometimes incomplete due to inadequate network design. Most importantly, there is not a large-scale real-world matting dataset, and current trimap-free methods trained with synthetic images suffer from large domain shift problems in practice. In this paper, we define the salient object as foreground, which is consistent with human cognition and annotations of the current matting dataset. Meanwhile, data and technologies in salient object detection can be transferred to matting in a breeze. To obtain a more accurate and complete alpha matte, we propose a network called \textbf{M}ulti-\textbf{F}eature fusion-based \textbf{C}oarse-to-fine Network \textbf{(MFC-Net)}, which fully integrates multiple features for an accurate and complete alpha matte. Furthermore, we introduce image harmony in data composition to bridge the gap between synthetic and real images. More importantly, we establish the largest general matting dataset \textbf{(Real-19k)} in the real world to date. Experiments show that our method is significantly effective on both synthetic and real-world images, and the performance in the real-world dataset is far better than existing matting-free methods. Our code and data will be released soon.

Figures (8)

• Figure 1: The bad case of existing trimap-free methods for extracting foreground. (a), (b), and (c) extract incorrect objects as the foreground. (d) preserves some visible noise as the foreground. (e) and (f) extract foreground obviously incomplete. (a), (b) and (c) is from modified Deeplab deeplab as discussed in boost. (d), (e), and (f) are from LFM lfm.
• Figure 2: Sample of the proposed Real-19k. Note that the annotation is alpha-level.
• Figure 3: Overview of the MFC-Net. The top shows the coarse module, which inputs the downsampled image and outputs the corresponding foreground. The bottom illustrates the refine module, which inputs upsampled foreground combined with the original image and outputs the alpha-level matting result. Numbers under the feature maps(like 256) are the numbers of channels.
• Figure 4: Overview of the IA. The input is $\boldsymbol{f}_{out}^{t}$, $\boldsymbol{f}_{g}^{t}$, and $\boldsymbol{f}_{l}^{t}$. $\boldsymbol{f}_{l}^{t}$ is the low-level appearance feature from encoder module via skip connection. $\boldsymbol{f}_{out}^{t}$ is high-level semantic feature and is the output of the HA or the previous IA. $\boldsymbol{f}_{g}^{t}$ is global context information which is the output of GCF. $t$ represents which stage it is, there are three stages as shown in Figure \ref{['pic_overview']}.
• Figure 5: The comparison between synthetic images without/with harmony. The left of the mask is without harmony, and the right is with harmony. It can be seen that 'harmony' adjusts the texture and brightness of the foreground, which adapts to different backgrounds, thereby making the synthetic image more natural.