Morpho-Aware Global Attention for Image Matting

TL;DR

This work proposes a novel Morpho-Aware Global Attention (MAGA) mechanism, designed to effectively capture the morphology of fine structures, and achieves significant performance gains, outperforming state-of-the-art methods across two benchmarks.

Abstract

Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) face inherent challenges in image matting, particularly in preserving fine structural details. ViTs, with their global receptive field enabled by the self-attention mechanism, often lose local details such as hair strands. Conversely, CNNs, constrained by their local receptive field, rely on deeper layers to approximate global context but struggle to retain fine structures at greater depths. To overcome these limitations, we propose a novel Morpho-Aware Global Attention (MAGA) mechanism, designed to effectively capture the morphology of fine structures. MAGA employs Tetris-like convolutional patterns to align the local shapes of fine structures, ensuring optimal local correspondence while maintaining sensitivity to morphological details. The extracted local morphology information is used as query embeddings, which are projected onto global key embeddings to emphasize local details in a broader context. Subsequently, by projecting onto value embeddings, MAGA seamlessly integrates these emphasized morphological details into a unified global structure. This approach enables MAGA to simultaneously focus on local morphology and unify these details into a coherent whole, effectively preserving fine structures. Extensive experiments show that our MAGA-based ViT achieves significant performance gains, outperforming state-of-the-art methods across two benchmarks with average improvements of 4.3% in SAD and 39.5% in MSE.

Figures (4)

• Figure 1: Overview of the proposed MAGA-based matting architecture. The framework input consists of the image combined with a grayscale trimap. A vision encoder, based on MAGA, extracts advanced semantics, while a simple CNN branch captures hierarchical low-level features, providing appearance cues. The advanced semantics are then progressively upsampled and fused with hierarchical low-level features through context fusion, ultimately producing a high-quality alpha matte.
• Figure 2: MAGA converts traditional patch embeddings from ViT architectures into 2D feature maps. Tetris-like convolutional kernels are applied to align the local shapes of fine structures. Through normalization and feature redistribution, MAGA employs a morpho-active learning mechanism to retain optimally matched local shapes, which are used as query embeddings. These shape-aware queries are progressively projected onto key and value embeddings, which carry rich global morphology, allowing local shapes to be contextually integrated within the global morphology. This process enhances local details and integrates them into a coherent whole.
• Figure 3: Comparisons with previous state-of-the-art methods on Adobe Composition-1k. Please zoom in for a clearer view of the details. MAGA demonstrates superior performance in preserving fine structural details.
• Figure 4: Evaluation on the number of MAGA blocks.