The Fairness-Quality Trade-off in Clustering

TL;DR

This paper addresses the fairness-quality trade-off in clustering by aiming to compute the entire Pareto front between clustering cost $c$ and a fairness objective $f$, rather than a single optimal point. It introduces a general framework based on pattern-based and mergeable fairness, and provides a dynamic-programming algorithm that exactly recovers the assignment front and a polynomial-time approach for the Sum of Imbalances when $l=2$, along with a $(2+\alpha,1)$-approximation for the clustering front. The authors establish hardness results explaining the inherent exponential complexity in the general case, while offering efficient solutions for key special cases and validating them on real datasets (Adult, Census1990, BlueBike). The work enables practitioners to explore the full spectrum of fairness-cost trade-offs, revealing actionable insights about how much quality to sacrifice to achieve desired fairness levels. Overall, the contributions advance fair clustering by delivering exact and approximate Pareto-front algorithms applicable to a wide range of objective combinations, with practical guidance and empirical demonstrations.

Abstract

Fairness in clustering has been considered extensively in the past; however, the trade-off between the two objectives -- e.g., can we sacrifice just a little in the quality of the clustering to significantly increase fairness, or vice-versa? -- has rarely been addressed. We introduce novel algorithms for tracing the complete trade-off curve, or Pareto front, between quality and fairness in clustering problems; that is, computing all clusterings that are not dominated in both objectives by other clusterings. Unlike previous work that deals with specific objectives for quality and fairness, we deal with all objectives for fairness and quality in two general classes encompassing most of the special cases addressed in previous work. Our algorithm must take exponential time in the worst case as the Pareto front itself can be exponential. Even when the Pareto front is polynomial, our algorithm may take exponential time, and we prove that this is inevitable unless P = NP. However, we also present a new polynomial-time algorithm for computing the entire Pareto front when the cluster centers are fixed, and for perhaps the most natural fairness objective: minimizing the sum, over all clusters, of the imbalance between the two groups in each cluster.

Figures (8)

• Figure 1: Pareto front recovered by Algorithm \ref{['alg:dynprogr']} for the Adult, Census, and BlueBike datasets (by row), for various fairness objectives (by column), for $k = 2$ clusters.
• Figure 2: Pareto front recovered by Algorithm \ref{['alg:dynprogr']} (labeled 'Dyn Progr', blue) and by Algorithm \ref{['alg:repeated_fcbc']} (labeled 'FCBC', orange), for $k = 2$ clusters.
• Figure 3: (a) An illustration of clustering under for non-pattern based fairness objectives. (b) An illustration of the $(P_i)_{i \in [8]}$ sets for non-mergeable fairness objectives.
• Figure 4: An illustration of implementing the repeated FCBC algorithm as the clustering cost upper bound U varies.
• Figure 5: Running time comparison with our dynamic programming approach from Algorithm \ref{['alg:dynprogr']}, labeled as 'Dyn Progr', and the repeated FCBC approach from Algorithm \ref{['alg:repeated_fcbc']}, labeled as 'FCBC', for each dataset (by column) and for the Group Utilitarian and Group Egalitarian objective (by row).

Theorems & Definitions (24)

• Definition 2.1: Pattern-based objectives
• Definition 2.2: Mergeable objectives
• Remark 3.1
• Theorem 3.2
• Definition 3.3: $\mathcal{W}$-approximation of the Pareto Set for clustering
• Theorem 3.4
• Theorem 3.5
• Theorem 3.6
• Remark 3.7
• proof : Proof of Theorem \ref{['thm:dynprogr-FA']}: