Transferring Fairness using Multi-Task Learning with Limited Demographic Information
Carlos Aguirre, Mark Dredze
TL;DR
The paper addresses fairness in NLP by transferring demographic fairness information across related tasks using a multi-task learning framework. It introduces a differentiable fairness objective called $\epsilon$-Differential Equalized Odds ($\epsilon$-DEO) and shows how to adapt single-task fairness losses to a multi-task setting to transfer fairness from an auxiliary task to a target task. Experiments across clinical notes from MIMIC-III, Trustpilot online reviews, and social media data (Twitter/HateXplain) demonstrate that MTL-fair can debias target tasks lacking demographic labels and can achieve intersectional fairness by combining single-axis attributes. Across domains, the approach yields fairer models with competitive accuracy, indicating that shared representations and joint optimization enable generalizable fairness without requiring demographics on every task.
Abstract
Training supervised machine learning systems with a fairness loss can improve prediction fairness across different demographic groups. However, doing so requires demographic annotations for training data, without which we cannot produce debiased classifiers for most tasks. Drawing inspiration from transfer learning methods, we investigate whether we can utilize demographic data from a related task to improve the fairness of a target task. We adapt a single-task fairness loss to a multi-task setting to exploit demographic labels from a related task in debiasing a target task and demonstrate that demographic fairness objectives transfer fairness within a multi-task framework. Additionally, we show that this approach enables intersectional fairness by transferring between two datasets with different single-axis demographics. We explore different data domains to show how our loss can improve fairness domains and tasks.