On Homomorphic Encryption Based Strategies for Class Imbalance in Federated Learning

TL;DR

FLICKER, a privacy preserving framework to address issues related to global class imbalance in federated learning, and shows that the proposed method improves the federated learning accuracy numbers by up to 8 when used along with popular datasets and relevant baselines.

Abstract

Class imbalance in training datasets can lead to bias and poor generalization in machine learning models. While pre-processing of training datasets can efficiently address both these issues in centralized learning environments, it is challenging to detect and address these issues in a distributed learning environment such as federated learning. In this paper, we propose FLICKER, a privacy preserving framework to address issues related to global class imbalance in federated learning. At the heart of our contribution lies the popular CKKS homomorphic encryption scheme, which is used by the clients to privately share their data attributes, and subsequently balance their datasets before implementing the FL scheme. Extensive experimental results show that our proposed method significantly improves the FL accuracy numbers when used along with popular datasets and relevant baselines.

Figures (8)

• Figure 1: FL setup with $3$ clients trying to solve $3$-class (circle, triangle and square) problem with class imbalance.
• Figure 2: Depiction of distributed computation of global distribution at the central server through homomorphic message-passing in the three clients model. Keys adjacent to the addition operator indicates homomorphic addition.
• Figure 3: Depiction of steps to localize the dominant client through projection operation in the three clients model.
• Figure 4: Depiction of local imbalance correction and subsequent update operation on global distribution in the three clients model. In this context $\overline{\mathbf{LD}_{2}}$ denotes the updated local distribution at the dominant client.
• Figure 5: FL accuracy comparison as a function of the training rounds when the initial distribution D1 of CIFAR datasets.