Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fair Distillation: Teaching Fairness from Biased Teachers in Medical Imaging

Milad Masroor, Tahir Hassan, Yu Tian, Kevin Wells, David Rosewarne, Thanh-Toan Do, Gustavo Carneiro

TL;DR

The Fair Distillation (FairDi) method is proposed, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group, and provides an effective solution for equitable model performance.

Abstract

Deep learning has achieved remarkable success in image classification and segmentation tasks. However, fairness concerns persist, as models often exhibit biases that disproportionately affect demographic groups defined by sensitive attributes such as race, gender, or age. Existing bias-mitigation techniques, including Subgroup Re-balancing, Adversarial Training, and Domain Generalization, aim to balance accuracy across demographic groups, but often fail to simultaneously improve overall accuracy, group-specific accuracy, and fairness due to conflicts among these interdependent objectives. We propose the Fair Distillation (FairDi) method, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group. These teacher models then guide the training of a unified ``student'' model, which distills their knowledge to maximize overall and group-specific accuracies, while minimizing inter-group disparities. Experiments on medical imaging datasets show that FairDi achieves significant gains in both overall and group-specific accuracy, along with improved fairness, compared to existing methods. FairDi is adaptable to various medical tasks, such as classification and segmentation, and provides an effective solution for equitable model performance.

Fair Distillation: Teaching Fairness from Biased Teachers in Medical Imaging

TL;DR

The Fair Distillation (FairDi) method is proposed, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group, and provides an effective solution for equitable model performance.

Abstract

Deep learning has achieved remarkable success in image classification and segmentation tasks. However, fairness concerns persist, as models often exhibit biases that disproportionately affect demographic groups defined by sensitive attributes such as race, gender, or age. Existing bias-mitigation techniques, including Subgroup Re-balancing, Adversarial Training, and Domain Generalization, aim to balance accuracy across demographic groups, but often fail to simultaneously improve overall accuracy, group-specific accuracy, and fairness due to conflicts among these interdependent objectives. We propose the Fair Distillation (FairDi) method, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group. These teacher models then guide the training of a unified ``student'' model, which distills their knowledge to maximize overall and group-specific accuracies, while minimizing inter-group disparities. Experiments on medical imaging datasets show that FairDi achieves significant gains in both overall and group-specific accuracy, along with improved fairness, compared to existing methods. FairDi is adaptable to various medical tasks, such as classification and segmentation, and provides an effective solution for equitable model performance.

Paper Structure

This paper contains 19 sections, 10 equations, 4 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Backbone Model Training (Step 0)
  5. Teacher Model Training (Step 1)
  6. Student Model Training (Step 2)
  7. Experiment
  8. Experimental Setup
  9. Classification Results
  10. Segmentation Results
  11. Ablation Study
  12. Training and Testing Running Times
  13. Conclusion
  14. Data Pre-processing
  15. Dataset Details
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Performance comparison of models (ERM vapnik1999overview, GroupDRO sagawa2019distributionally, SWAD cha2021swad, FIS luo2024fairvisionequitabledeeplearning, and our FairDi) on the HAM10000 dataset tschandl2018ham10000 for benign/malignant classification by gender. The left panel shows group-specific AUCs (Male and Female), and the right panel plots fairness (AUC Gap) vs. Overall AUC. Each model's Pareto front includes two points: one maximizing worst-group AUC, the other maximizing overall AUC. Our FairDi achieves the best balance with high overall AUC, low AUC Gap, and robust group-specific AUCs.
  • Figure 2: Diagram of the FairDi training process. After pre-training (step 0), it optimizes the teachers' performances for their respective sensitive groups (step 1), followed by a knowledge distillation process that trains a single student to optimize overall and group-specific accuracies, while minimizing performance gap between groups (step 2). The flame symbol indicates model components to be trained during each corresponding step, while the snowflake symbol represents components that remain frozen.
  • Figure 3: Performance of fairness algorithms for classification across all datasets as average rank CD diagrams. Our FairDi is the highest ranked method for all settings, being significantly better than most methods, and for the worst-case AUC, it is the single best method.
  • Figure 4: Performance of fairness segmentation algorithms shown with average rank CD diagrams. FairDi consistently outperforms most methods, ranking the highest in all settings. In this figure, TUN stands for TransUNet