Mask-Guided Multi-Task Network for Face Attribute Recognition

TL;DR

The paper addresses face attribute recognition (FAR) within a multi-task learning framework, where global feature usage can cause feature redundancy and negative transfer. It introduces the Mask-Guided Multi-Task Network (MGMTN), which leverages Adaptive Mask Learning (AML) to localize informative facial parts and Group-Global Feature Fusion (G2FF) to blend region-level and global cues. AML uses a UNet trained on FaRL-based keypoint annotations to produce eight group masks, enabling region-aware feature learning, while G2FF refines and fuses these group features with global features for robust attribute prediction. On CelebA and LFWA, MGMTN achieves state-of-the-art results and demonstrates strong part-segmentation performance (F1 around 89%), highlighting the value of region-aware learning for FAR without requiring additional pixel-level supervision during inference.

Abstract

Face Attribute Recognition (FAR) plays a crucial role in applications such as person re-identification, face retrieval, and face editing. Conventional multi-task attribute recognition methods often process the entire feature map for feature extraction and attribute classification, which can produce redundant features due to reliance on global regions. To address these challenges, we propose a novel approach emphasizing the selection of specific feature regions for efficient feature learning. We introduce the Mask-Guided Multi-Task Network (MGMTN), which integrates Adaptive Mask Learning (AML) and Group-Global Feature Fusion (G2FF) to address the aforementioned limitations. Leveraging a pre-trained keypoint annotation model and a fully convolutional network, AML accurately localizes critical facial parts (e.g., eye and mouth groups) and generates group masks that delineate meaningful feature regions, thereby mitigating negative transfer from global region usage. Furthermore, G2FF combines group and global features to enhance FAR learning, enabling more precise attribute identification. Extensive experiments on two challenging facial attribute recognition datasets demonstrate the effectiveness of MGMTN in improving FAR performance.

Figures (9)

• Figure 1: (a) The conventional global-based approach prior to multi-task attribute recognition. (b) The MGMTN approach, which leverages masks to guide group feature learning, effectively reduces feature redundancy typical of global-based methods. Global layers extract holistic features, while UNet generates group features from low-level feature maps. Here, $\otimes$ denotes element-wise multiplication.
• Figure 2: (a) A $3\times3$ convolution generates a feature map by aggregating information from 9 neighboring pixels into a single feature value. (b) In ResNet50 or ResNeSt50, features extracted by a convolutional layer exhibit decreasing similarity with increasing spatial distance from each location. (c) Comparison across various facial parts and object locations, retaining only the regions of interest.
• Figure 3: Using keypoint annotations, rectangular contours are generated to delineate individual facial groups. Each mask highlights the corresponding group region with white pixels, while all other pixels represent the background.
• Figure 4: The MGMTN architecture incorporates multiple groups, with some processing the entire feature map and others extracting specific group features. AML is employed to identify informative regions via group masks, which are then used to learn group features. These features are fused through the G2FF mechanism to produce the final group attribute representation. Each classification head consists of two fully connected layers: the first reduces the dimension of the enhanced group features $\rm{G}$ from 3584 to 512, and the second independently predicts attribute values for each group feature. Here, $\otimes$ denotes element-wise multiplication, and $\oplus$ denotes channel-wise concatenation.
• Figure 5: Illustration of the AML process using UNet. The final convolutional layer of UNet consists of eight convolutional heads, each independently predicting a group mask.