Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Heterogeneous Performance-Fairness Trade-offs in Federated Learning

Rongguang Ye, Ming Tang

TL;DR

HetPFL tackles fairness-aware federated learning in the presence of heterogeneous client trade-offs by learning both heterogeneous local Pareto fronts and a strong global Pareto front. It combines Preference Sampling Adaptation (PSA), which adapts the per-client preference distribution using Hypervolume Contribution, with Preference-aware Hypernet Fusion (PHF), which learns how to fuse client hypernets at the server via FusionNet for each preference. The approach comes with a convergence guarantee of order $\mathcal{O}\left(\tfrac{1}{t}\right)$ and demonstrates clear improvements in both local and global Pareto front quality over strong baselines across four datasets, including large-scale client scenarios. The framework offers a practical, privacy-preserving way to tailor fairness-performance trade-offs across heterogeneous FL deployments and to generalize well to the global dataset.

Abstract

Recent methods leverage a hypernet to handle the performance-fairness trade-offs in federated learning. This hypernet maps the clients' preferences between model performance and fairness to preference-specifc models on the trade-off curve, known as local Pareto front. However, existing methods typically adopt a uniform preference sampling distribution to train the hypernet across clients, neglecting the inherent heterogeneity of their local Pareto fronts. Meanwhile, from the perspective of generalization, they do not consider the gap between local and global Pareto fronts on the global dataset. To address these limitations, we propose HetPFL to effectively learn both local and global Pareto fronts. HetPFL comprises Preference Sampling Adaptation (PSA) and Preference-aware Hypernet Fusion (PHF). PSA adaptively determines the optimal preference sampling distribution for each client to accommodate heterogeneous local Pareto fronts. While PHF performs preference-aware fusion of clients' hypernets to ensure the performance of the global Pareto front. We prove that HetPFL converges linearly with respect to the number of rounds, under weaker assumptions than existing methods. Extensive experiments on four datasets show that HetPFL significantly outperforms seven baselines in terms of the quality of learned local and global Pareto fronts.

Learning Heterogeneous Performance-Fairness Trade-offs in Federated Learning

TL;DR

HetPFL tackles fairness-aware federated learning in the presence of heterogeneous client trade-offs by learning both heterogeneous local Pareto fronts and a strong global Pareto front. It combines Preference Sampling Adaptation (PSA), which adapts the per-client preference distribution using Hypervolume Contribution, with Preference-aware Hypernet Fusion (PHF), which learns how to fuse client hypernets at the server via FusionNet for each preference. The approach comes with a convergence guarantee of order and demonstrates clear improvements in both local and global Pareto front quality over strong baselines across four datasets, including large-scale client scenarios. The framework offers a practical, privacy-preserving way to tailor fairness-performance trade-offs across heterogeneous FL deployments and to generalize well to the global dataset.

Abstract

Recent methods leverage a hypernet to handle the performance-fairness trade-offs in federated learning. This hypernet maps the clients' preferences between model performance and fairness to preference-specifc models on the trade-off curve, known as local Pareto front. However, existing methods typically adopt a uniform preference sampling distribution to train the hypernet across clients, neglecting the inherent heterogeneity of their local Pareto fronts. Meanwhile, from the perspective of generalization, they do not consider the gap between local and global Pareto fronts on the global dataset. To address these limitations, we propose HetPFL to effectively learn both local and global Pareto fronts. HetPFL comprises Preference Sampling Adaptation (PSA) and Preference-aware Hypernet Fusion (PHF). PSA adaptively determines the optimal preference sampling distribution for each client to accommodate heterogeneous local Pareto fronts. While PHF performs preference-aware fusion of clients' hypernets to ensure the performance of the global Pareto front. We prove that HetPFL converges linearly with respect to the number of rounds, under weaker assumptions than existing methods. Extensive experiments on four datasets show that HetPFL significantly outperforms seven baselines in terms of the quality of learned local and global Pareto fronts.
Paper Structure (25 sections, 4 theorems, 32 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 25 sections, 4 theorems, 32 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Methodology
  4. Overview
  5. Phase (I)
  6. Phase (II)
  7. Preference Sampling Adaptation
  8. Preference-aware Hypernet Fusion
  9. Algorithm: HetPFL
  10. Theoretical Analysis
  11. Experiments
  12. Experimental Settings
  13. Experimental Results
  14. Conclusion
  15. Schematic diagram
  16. ...and 10 more sections

Key Result

Lemma 1

Given a preference vector $\boldsymbol{\lambda}$, a preference-specific model $h_{\boldsymbol{\beta}_{k}}(\boldsymbol{\lambda})$ is weakly Pareto optimal to the problem (tch) if and only if $h_{\boldsymbol{\beta}_{k}}(\boldsymbol{\lambda})$ is optimal for problem (tch).

Figures (9)

  • Figure 1: The impact of different sampling distributions under two clients. The dotted vectors represent preferences for model's performance and fairness. The pink and green points are loss vectors of the model after evaluation on the local dataset on client (a) and client (b), respectively. A uniform preference sampling distribution cannot achieve the best result of learning local Pareto fronts based on Lemma \ref{['prop']}. Instead, sampling distribution (I) is suitable for client (a), and sampling distribution (II) is suitable for client (b).
  • Figure 2: HetPFL framework.
  • Figure 3: An illustration of the HV and HVC of the loss vectors $\boldsymbol{\ell}^{(i)}$ produced by evaluating a model set $\{f_{\boldsymbol{\theta}_k \mid {\boldsymbol{\lambda}^{i}}}\}_{i=1}^{5}$ corresponding to five input preference vectors.
  • Figure 4: Comparison of global Pareto front obtained by our HetPFL algorithm and baselines on four datasets. A Pareto front closer to the bottom-left corner indicates better performance.
  • Figure 5: Convergence of HetPFL compared with PraFFL.
  • ...and 4 more figures

Theorems & Definitions (5)

  • Lemma 1: Preference Alignment
  • Definition 1: HVC
  • Lemma 2: Convergence of the Communicated Model collins2021exploiting
  • Theorem 1: Convergence of the Hypernet
  • Lemma 3: Convergence of the Hypernet and Sampling Distribution in One Round hong2023two