FairDD: Fair Dataset Distillation

TL;DR

This work proposes a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures, and significantly improves fairness compared to vanilla DDs.

Abstract

Condensing large datasets into smaller synthetic counterparts has demonstrated its promise for image classification. However, previous research has overlooked a crucial concern in image recognition: ensuring that models trained on condensed datasets are unbiased towards protected attributes (PA), such as gender and race. Our investigation reveals that dataset distillation fails to alleviate the unfairness towards minority groups within original datasets. Moreover, this bias typically worsens in the condensed datasets due to their smaller size. To bridge the research gap, we propose a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures. The key innovation of FairDD lies in synchronously matching synthetic datasets to PA-wise groups of original datasets, rather than indiscriminate alignment to the whole distributions in vanilla DDs, dominated by majority groups. This synchronized matching allows synthetic datasets to avoid collapsing into majority groups and bootstrap their balanced generation to all PA groups. Consequently, FairDD could effectively regularize vanilla DDs to favor biased generation toward minority groups while maintaining the accuracy of target attributes. Theoretical analyses and extensive experimental evaluations demonstrate that FairDD significantly improves fairness compared to vanilla DDs, with a promising trade-off between fairness and accuracy. Its consistent superiority across diverse DDs, spanning Distribution and Gradient Matching, establishes it as a versatile FDD approach. Code is available at https://github.com/zqhang/FairDD.

Figures (17)

• Figure 1: Visualization comparison on $\mathcal{S}$ at IPC = 10 for diverse datasets. FairDD successfully mitigates the bias from original datasets in (a) foreground digital color, (b) background color, (c) foreground and background grayscale (d) real-world bias. (e) vanilla DDs exacerbate the unfairness.
• Figure 1: Fairness comparison on diverse IPCs.
• Figure 2: The overview of FairDD. FairDD first groups target signals of $\mathcal{T}$ and then proposes to align $\mathcal{S}$ (random initialization) with respective group centers. With this synchronized matching, $\mathcal{S}$ is simultaneously pulled by all group centers in a batch. This prevents the condensed dataset $\mathcal{S}$ from being biased towards the majority group, allowing it to better cover the distribution of $\mathcal{T}$.
• Figure 2: Accuracy comparison on diverse IPCs.
• Figure 3: T-SNE visualization towards test features. Color represents distinct PA groups in (a) and (b), and TA labels in (c) and (d). In (a), DM shows obvious distinctiveness towards different PA. But (b) shows DM+FairDD eliminates the recognition of PA. In (c) and (d), DM+FairDD enables compact TA representations, but DM tends to cluster features with the same PA.

Theorems & Definitions (4)

• Theorem 5.1
• proof
• Theorem 5.2
• Theorem D.1