Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous Fair SMOTE -- Fairness-Aware Stream Learning from Imbalanced Data

Kathrin Lammers, Valerie Vaquet, Barbara Hammer

TL;DR

CFSMOTE addresses fairness under class-imbalance in online data streams by extending continuous SMOTE with situation testing and four-group balancing across class and sensitive attributes. It operates as a pre-processing step (classifier-agnostic) and uses ADWIN drift detectors to manage windows and trigger oversampling without storing synthetic samples. Empirical results on Adult and KDD Census streams show substantial improvements in group fairness metrics such as statistical parity, disparate impact, equal opportunity, and equal FPR, with competitive or modest trade-offs in predictive performance relative to C-SMOTE and to fairness-aware stream learners FABBOO and FAHT. The work also highlights the need for appropriate streaming fairness benchmarks and suggests directions for future extensions.

Abstract

As machine learning is increasingly applied in an online fashion to deal with evolving data streams, the fairness of these algorithms is a matter of growing ethical and legal concern. In many use cases, class imbalance in the data also needs to be dealt with to ensure predictive performance. Current fairness-aware stream learners typically attempt to solve these issues through in- or post-processing by focusing on optimizing one specific discrimination metric, addressing class imbalance in a separate processing step. While C-SMOTE is a highly effective model-agnostic pre-processing approach to mitigate class imbalance, as a side effect of this method, algorithmic bias is often introduced. Therefore, we propose CFSMOTE - a fairness-aware, continuous SMOTE variant - as a pre-processing approach to simultaneously address the class imbalance and fairness concerns by employing situation testing and balancing fairness-relevant groups during oversampling. Unlike other fairness-aware stream learners, CFSMOTE is not optimizing for only one specific fairness metric, therefore avoiding potentially problematic trade-offs. Our experiments show significant improvement on several common group fairness metrics in comparison to vanilla C-SMOTE while maintaining competitive performance, also in comparison to other fairness-aware algorithms.

Continuous Fair SMOTE -- Fairness-Aware Stream Learning from Imbalanced Data

TL;DR

CFSMOTE addresses fairness under class-imbalance in online data streams by extending continuous SMOTE with situation testing and four-group balancing across class and sensitive attributes. It operates as a pre-processing step (classifier-agnostic) and uses ADWIN drift detectors to manage windows and trigger oversampling without storing synthetic samples. Empirical results on Adult and KDD Census streams show substantial improvements in group fairness metrics such as statistical parity, disparate impact, equal opportunity, and equal FPR, with competitive or modest trade-offs in predictive performance relative to C-SMOTE and to fairness-aware stream learners FABBOO and FAHT. The work also highlights the need for appropriate streaming fairness benchmarks and suggests directions for future extensions.

Abstract

As machine learning is increasingly applied in an online fashion to deal with evolving data streams, the fairness of these algorithms is a matter of growing ethical and legal concern. In many use cases, class imbalance in the data also needs to be dealt with to ensure predictive performance. Current fairness-aware stream learners typically attempt to solve these issues through in- or post-processing by focusing on optimizing one specific discrimination metric, addressing class imbalance in a separate processing step. While C-SMOTE is a highly effective model-agnostic pre-processing approach to mitigate class imbalance, as a side effect of this method, algorithmic bias is often introduced. Therefore, we propose CFSMOTE - a fairness-aware, continuous SMOTE variant - as a pre-processing approach to simultaneously address the class imbalance and fairness concerns by employing situation testing and balancing fairness-relevant groups during oversampling. Unlike other fairness-aware stream learners, CFSMOTE is not optimizing for only one specific fairness metric, therefore avoiding potentially problematic trade-offs. Our experiments show significant improvement on several common group fairness metrics in comparison to vanilla C-SMOTE while maintaining competitive performance, also in comparison to other fairness-aware algorithms.
Paper Structure (12 sections, 3 figures, 4 tables)

This paper contains 12 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Foundations
  3. Learning from Data Streams
  4. Fairness in Machine Learning
  5. Related Work
  6. CFSMOTE
  7. Experiments
  8. Datasets
  9. Setup
  10. CFSMOTE vs CSMOTE
  11. CFSMOTE vs Fairness-Aware Stream Learners
  12. Conclusion

Figures (3)

  • Figure 1: An overview of the CFSMOTE algorithm. A new sample must pass situation testing before it gets passed to the base learner. Simultaneously, it is used to update the window of current samples. If imbalance is detected, SMOTE-based oversampling for the minority group is triggered. Synthetic samples do not get stored, but the imbalance rates get updated after generation, and the samples get passed on to situation testing. The window of samples is constantly monitored by an ADWIN drift detector and adjusted accordingly.
  • Figure 2: CFSMOTE vs C-SMOTE
  • Figure 3: Comparison of Fair Learners