Multi-Task Learning Using Uncertainty to Weigh Losses for Heterogeneous Face Attribute Estimation

TL;DR

A generalized framework for joint estimation of ordinal and nominal attributes based on information sharing is proposed and Experimental results on benchmarks with multiple face attributes show that the proposed approach has superior performance compared to state of the art.

Abstract

Face images contain a wide variety of attribute information. In this paper, we propose a generalized framework for joint estimation of ordinal and nominal attributes based on information sharing. We tackle the correlation problem between heterogeneous attributes using hard parameter sharing of shallow features, and trade-off multiple loss functions by considering homoskedastic uncertainty for each attribute estimation task. This leads to optimal estimation of multiple attributes of the face and reduces the training cost of multitask learning. Experimental results on benchmarks with multiple face attributes show that the proposed approach has superior performance compared to state of the art. Finally, we discuss the bias issues arising from the proposed approach in face attribute estimation and validate its feasibility on edge systems.

Figures (7)

• Figure 1: Individual face attributes have both correlation and heterogeneity.
• Figure 2: Overview of the proposed DMTL approach consisting of an early-stage shared feature learning for all the attributes, followed by category-related feature learning for heterogeneous attribute categories. We use the MBConvtan2019efficientnet for shared feature learning. Each task achieves optimal estimation of individual heterogeneous attributes through fine-tuning, e.g., nominal versus ordinal.
• Figure 3: Convergence curves of task weights training on two datasets, $\beta_i$ represents the weight of the attribute $i$ estimation task in the Joint loss $\mathcal{L}$. (a) Adience. (b) UTKFace.
• Figure 4: The comparison of age estimation with CS metric on UTKFace benchmark.
• Figure 5: Visualizing explanations using Grad-CAM for the proposed approach.'a/b' denotes the predicted/labeled information of each image, with 'M/F' denoting male/female. (a) Age:1/1; (b) Gender:M/M; (c) Race:White/White; (d) Age:37/36; (e) Gender:M/M; (f) Race:Indian/Indian; (g) Age:42/60; (h) Gender:F/F; (i) Race:White/White.