Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching

TL;DR

This work tackles the dual challenge of personalization and extreme communication efficiency in federated learning by introducing pFed1BS, a framework that uses one-bit random sketches for both uplink updates and downlink consensus. It formulates a principled bilevel optimization with a sign-based regularizer and employs a fast structured projection via the Fast Hadamard Transform to enable scalable, bidirectional one-bit communication. Theoretical analysis shows convergence to a stationary neighborhood of a global potential under standard FL assumptions, while empirical results on MNIST, FMNIST, CIFAR-10/100, and SVHN demonstrate substantial communication cost reductions (over 96%) with competitive accuracy compared to state-of-the-art baselines. The approach significantly advances practical personalized FL in bandwidth-limited environments, offering a robust balance between personalization quality and communication efficiency.

Abstract

Federated Learning (FL) enables collaborative training across decentralized data, but faces key challenges of bidirectional communication overhead and client-side data heterogeneity. To address communication costs while embracing data heterogeneity, we propose pFed1BS, a novel personalized federated learning framework that achieves extreme communication compression through one-bit random sketching. In personalized FL, the goal shifts from training a single global model to creating tailored models for each client. In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus. To enable effective personalization, we introduce a sign-based regularizer that guides local models to align with the global consensus while preserving local data characteristics. To mitigate the computational burden of random sketching, we employ the Fast Hadamard Transform for efficient projection. Theoretical analysis guarantees that our algorithm converges to a stationary neighborhood of the global potential function. Numerical simulations demonstrate that pFed1BS substantially reduces communication costs while achieving competitive performance compared to advanced communication-efficient FL algorithms.

Figures (7)

• Figure 1: An overview of our proposed framework. At round $t$, each client performs a local update using a sign-based regularizer with the global one-bit vector $\bm{v}^t$. Then each client projects and quantizes the updated local model to one bit vector $\hbox{sign}(\mathbf{\Phi} \bm{w}_k^{t+1})$, and then transmits it to the server. The server aggregates all clients' one-bit vectors to form the next global one-bit vector $\bm{v}^{t+1}$, which is broadcast for the next round.
• Figure 1: Performance of pFed1BS with a varying number of participating clients ($S$) on MNIST.
• Figure 2: Comparison between (a) a dense random projection and (b) our efficient structured projection. The structured projection sequentially applies element-wise random sign flips ($\bm{D}$), a Fast Hadamard Transform ($\bm{H}$), and random subsampling ($\bm{S}$).
• Figure 2: Effect of the number of local epochs ($R$) on MNIST.
• Figure 3: Test accuracy on MNIST (non-i.i.d.). pFed1BS achieves both faster convergence and higher final accuracy.

Theorems & Definitions (28)

• Lemma 1: Optimal Server Aggregation
• Lemma 2: Bounded Projection Norm
• Lemma 3: Client-Side Objective and Gradient
• Lemma 4: Smoothness of Client Objective
• Lemma 5: Bounded Model Norm
• Lemma 6: Variance of Client Sampling
• Lemma 7: Client-Side Objective Descent
• Theorem 1: Local Convergence
• Remark 1
• Remark 2