Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedPref: Federated Learning Across Heterogeneous Multi-objective Preferences

Maria Hartmann, Grégoire Danoy, Pascal Bouvry

TL;DR

FedPref addresses objective-heterogeneous federated learning by learning personalised models that reflect each client's multi-objective preferences. It combines a modified cosine-based similarity metric with recursive clustering and weighted aggregation to group compatible clients while maintaining individualisation, and it validates performance across diverse MORL tasks. Beyond client-level gains, the paper introduces a multi-objective evaluation framework showing FedPref yields diverse, convergent trade-offs that better capture real-world preferences. Overall, FedPref demonstrates robust adaptability to different preference distributions and problem characteristics, offering a privacy-preserving, scalable approach to personalised FL in multi-objective settings.

Abstract

Federated Learning (FL) is a distributed machine learning strategy, developed for settings where training data is owned by distributed devices and cannot be shared. FL circumvents this constraint by carrying out model training in distribution. The parameters of these local models are shared intermittently among participants and aggregated to enhance model accuracy. This strategy has been rapidly adopted by the industry in efforts to overcome privacy and resource constraints in model training. However, the application of FL to real-world settings brings additional challenges associated with heterogeneity between participants. Research into mitigating these difficulties in FL has largely focused on only two types of heterogeneity: the unbalanced distribution of training data, and differences in client resources. Yet more types of heterogeneity are becoming relevant as the capability of FL expands to cover more complex problems, from the tuning of LLMs to enabling machine learning on edge devices. In this work, we discuss a novel type of heterogeneity that is likely to become increasingly relevant in future applications: this is preference heterogeneity, emerging when clients learn under multiple objectives, with different importance assigned to each objective on different clients. In this work, we discuss the implications of this type of heterogeneity and propose FedPref, a first algorithm designed to facilitate personalised FL in this setting. We demonstrate the effectiveness of the algorithm across different problems, preference distributions and model architectures. In addition, we introduce a new analytical point of view, based on multi-objective metrics, for evaluating the performance of FL algorithms in this setting beyond the traditional client-focused metrics. We perform a second experimental analysis based in this view, and show that FedPref outperforms compared algorithms.

FedPref: Federated Learning Across Heterogeneous Multi-objective Preferences

TL;DR

FedPref addresses objective-heterogeneous federated learning by learning personalised models that reflect each client's multi-objective preferences. It combines a modified cosine-based similarity metric with recursive clustering and weighted aggregation to group compatible clients while maintaining individualisation, and it validates performance across diverse MORL tasks. Beyond client-level gains, the paper introduces a multi-objective evaluation framework showing FedPref yields diverse, convergent trade-offs that better capture real-world preferences. Overall, FedPref demonstrates robust adaptability to different preference distributions and problem characteristics, offering a privacy-preserving, scalable approach to personalised FL in multi-objective settings.

Abstract

Federated Learning (FL) is a distributed machine learning strategy, developed for settings where training data is owned by distributed devices and cannot be shared. FL circumvents this constraint by carrying out model training in distribution. The parameters of these local models are shared intermittently among participants and aggregated to enhance model accuracy. This strategy has been rapidly adopted by the industry in efforts to overcome privacy and resource constraints in model training. However, the application of FL to real-world settings brings additional challenges associated with heterogeneity between participants. Research into mitigating these difficulties in FL has largely focused on only two types of heterogeneity: the unbalanced distribution of training data, and differences in client resources. Yet more types of heterogeneity are becoming relevant as the capability of FL expands to cover more complex problems, from the tuning of LLMs to enabling machine learning on edge devices. In this work, we discuss a novel type of heterogeneity that is likely to become increasingly relevant in future applications: this is preference heterogeneity, emerging when clients learn under multiple objectives, with different importance assigned to each objective on different clients. In this work, we discuss the implications of this type of heterogeneity and propose FedPref, a first algorithm designed to facilitate personalised FL in this setting. We demonstrate the effectiveness of the algorithm across different problems, preference distributions and model architectures. In addition, we introduce a new analytical point of view, based on multi-objective metrics, for evaluating the performance of FL algorithms in this setting beyond the traditional client-focused metrics. We perform a second experimental analysis based in this view, and show that FedPref outperforms compared algorithms.
Paper Structure (33 sections, 5 equations, 24 figures, 14 tables, 3 algorithms)

This paper contains 33 sections, 5 equations, 24 figures, 14 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Federated multi-objective learning
  4. Federated multi-task learning
  5. Other types of federated heterogeneity
  6. The FedPref algorithm
  7. Problem formulation
  8. Concept sketch and definitions
  9. Similarity metric
  10. Weighted aggregation
  11. Recursive clustering
  12. Full algorithm
  13. Client-level evaluation
  14. Implementation and setup
  15. Comparison to baselines
  16. ...and 18 more sections

Figures (24)

  • Figure 1: Different preferences lead to different solutions in a yellow submarine searching for underwater treasure. Left: A strong preference for minimising travel distance. Centre: Balanced preferences. Right: A strong preference for maximising the value of the treasure reward. The goal of our work is to allow clients with problems like these to perform FL effectively, despite their heterogeneous objective preferences.
  • Figure 2: An illustration of the federated system solving a multi-objective problem. In this instance, we want to learn to plan trajectories for drones, under two potentially conflicting objectives: conserving energy and maximising speed. Each drone assigns different importance (preference weights) to these objectives. Federated Learning takes place as follows: (1) Clients (drones) perform local training, using the objective function defined by their preferences. (2) Clients submit model updates to the server. (3) The server aggregates these model updates, obtaining personalised models. (4) The server returns the respective personalised models to the clients.
  • Figure 3: A schematic representation of the flow between components of the algorithm.
  • Figure 4: Geometric interpretation of cosine similarity.
  • Figure 5: A weighted aggregation step inside a single cluster. Left: personalised client updates are computed using aggregation weights based on client similarity relative to the cluster-mean. Right: The updated cluster-mean is computed.
  • ...and 19 more figures