PraFFL: A Preference-Aware Scheme in Fair Federated Learning

TL;DR

PraFFL tackles the fairness-performance trade-off in federated learning by introducing a preference-aware framework that supports arbitrary client preferences in real time. It aligns models to user-specified trade-offs using a weighted Chebyshev objective, personalizes models via a two-part shared/personalized structure, and protects client preferences with a hypernetwork. Theoretical results establish existence and linear convergence to the Pareto front, while empirical results on four datasets show superior Pareto front quality and adaptability against six baselines, with code available for reproduction. This approach enables scalable, privacy-preserving, and customizable fair FL suitable for large-scale deployments and diverse user needs.

Abstract

Fairness in federated learning has emerged as a critical concern, aiming to develop an unbiased model among groups (e.g., male or female) of diverse sensitive features. However, there is a trade-off between model performance and fairness, i.e., improving model fairness will decrease model performance. Existing approaches have characterized such a trade-off by introducing hyperparameters to quantify client's preferences for model fairness and model performance. Nevertheless, these approaches are limited to scenarios where each client has only a single pre-defined preference, and fail to work in practical systems where each client generally has multiple preferences. To this end, we propose a Preference-aware scheme in Fair Federated Learning (called PraFFL) to generate preference-specific models in real time. PraFFL can adaptively adjust the model based on each client's preferences to meet their needs. We theoretically prove that PraFFL can offer the optimal model tailored to an arbitrary preference of each client, and show its linear convergence. Experimental results show that our proposed PraFFL outperforms six fair federated learning algorithms in terms of the model's capability of adapting to clients' different preferences. Our implementation is available at https://github.com/rG223/PraFFL.

Figures (9)

• Figure 1: The model trained under different approaches. Gray stars represent the desired models, and colored circles represent the obtained models. The red curve indicates the optimal performance-fairness trade-offs, typically unknown in advance. $\delta^*$ is the fair budget that most closely aligns with the green preference arrow. For the first case, the desired model is obtained when $\delta=\delta^*$, where $\delta^*$ is the fair budget that most closely aligns with the green preference arrow. However, since the Pareto front is unknown, it is not possible to determine the exact value of $\delta^*$ that corresponds to the intersection of the preference direction and the Pareto front. Instead, $\delta$ must be repeatedly adjusted until it reaches $\delta^*$. For the second case, the obtained model deviates from the desired model due to the concave shape of the Pareto front (see Fig. \ref{['fig:tch']}). For the third case, our proposed approach not only achieves the desired models but also enables simultaneous handling of multiple preferences.
• Figure 2: Diagram of hypervolume.
• Figure 3: The solution obtained by different optimization functions. (a): The weighted-sum function (Eq. (\ref{['basic']})). (b): The weighted Tchebycheff function (Eq. (\ref{['tch_fl']})).
• Figure 4: Personalized federated learning framework.
• Figure 5: The illustration of hypernetwork inference for client $k$. Hypernetwork ${\bm{\beta}_{k}}$ returns the corresponding personalized models to client $k$ based on the preference vectors.

Theorems & Definitions (13)

• Definition 3.1: Pareto Dominance
• Definition 3.2: Pareto Optimality
• Definition 3.3: Pareto Set/Front
• Definition 3.4: Hypervolume
• lemma 1: Preference Alignment miettinen1999nonlinear
• lemma 2: Existence
• lemma 3: Pareto Optimality
• theorem 1: Pareto Front
• lemma 4: Convexity of Smooth Tchebycheff Function
• lemma 5: Smoothness of Smooth Tchebycheff Function