Voronoi-grid-based Pareto Front Learning and Its Application to Collaborative Federated Learning
Mengmeng Chen, Xiaohu Wu, Qiqi Liu, Tiantian He, Yew-Soon Ong, Yaochu Jin, Qicheng Lao, Han Yu
TL;DR
This work tackles the limitations of existing Pareto Front Learning methods in high-dimensional spaces by introducing a Voronoi-grid-based sampling paradigm (PHN-HVVS) that uses a GA to partition the design space into Voronoi grids. A novel loss combines Hypervolume (HV) maximization with a distance-based penalty to ensure full Pareto front coverage, including boundary regions, and to push solutions toward the true front. The approach is instantiated via Pareto HyperNetworks (PHNs) that map preference vectors to model parameters, enabling flexible exploration of trade-offs. Empirical results across toy MO scenarios, multi-task learning, and collaborative federated learning demonstrate superior HV and more comprehensive front coverage, with robust performance under diverse problem shapes (convex/concave) and data heterogeneity. The work advances scalable MOO in ML and provides a practical pathway to more effective collaboration strategies in FL through better front optimization and benefit graph generation.
Abstract
Multi-objective optimization (MOO) exists extensively in machine learning, and aims to find a set of Pareto-optimal solutions, called the Pareto front, e.g., it is fundamental for multiple avenues of research in federated learning (FL). Pareto-Front Learning (PFL) is a powerful method implemented using Hypernetworks (PHNs) to approximate the Pareto front. This method enables the acquisition of a mapping function from a given preference vector to the solutions on the Pareto front. However, most existing PFL approaches still face two challenges: (a) sampling rays in high-dimensional spaces; (b) failing to cover the entire Pareto Front which has a convex shape. Here, we introduce a novel PFL framework, called as PHN-HVVS, which decomposes the design space into Voronoi grids and deploys a genetic algorithm (GA) for Voronoi grid partitioning within high-dimensional space. We put forward a new loss function, which effectively contributes to more extensive coverage of the resultant Pareto front and maximizes the HV Indicator. Experimental results on multiple MOO machine learning tasks demonstrate that PHN-HVVS outperforms the baselines significantly in generating Pareto front. Also, we illustrate that PHN-HVVS advances the methodologies of several recent problems in the FL field. The code is available at https://github.com/buptcmm/phnhvvs}{https://github.com/buptcmm/phnhvvs.