Federated Linear Contextual Bandits with Heterogeneous Clients

TL;DR

This work extends federated bandit learning to heterogeneous client populations by clustering clients and performing cluster-wise collaborative learning under a standard FL single-model broadcast. The HetoFedBandit framework comprises a pure-exploration phase for clustering, followed by an optimistic learning phase that leverages cross-client information within identified clusters, all coordinated via a FIFO cluster-level communication protocol. The authors establish clustering correctness with high probability, derive confidence ellipsoids, and bound both regret and communication cost, while also proposing empirical enhancements (data-dependent clustering and a priority queue) that improve performance in practice. Experiments on synthetic and LastFM datasets demonstrate that HetoFedBandit and its enhanced variant achieve sub-linear regret and lower communication costs compared to strong baselines, highlighting the approach's practical potential for private, distributed bandit learning with heterogeneity.

Abstract

The demand for collaborative and private bandit learning across multiple agents is surging due to the growing quantity of data generated from distributed systems. Federated bandit learning has emerged as a promising framework for private, efficient, and decentralized online learning. However, almost all previous works rely on strong assumptions of client homogeneity, i.e., all participating clients shall share the same bandit model; otherwise, they all would suffer linear regret. This greatly restricts the application of federated bandit learning in practice. In this work, we introduce a new approach for federated bandits for heterogeneous clients, which clusters clients for collaborative bandit learning under the federated learning setting. Our proposed algorithm achieves non-trivial sub-linear regret and communication cost for all clients, subject to the communication protocol under federated learning that at anytime only one model can be shared by the server.

Figures (4)

• Figure 1: Execution of HetoFedBandit after pure exploration phase $T_0$. Each client $i \in N$ is represented by a node in the client graph $\mathcal{G}$ on the left hand side. Edges between clients indicate potential collaborators, as defined by the homogeneity test. The colored ellipsoids represent the estimated clusters $\hat{\mathcal{C}} = \{\hat{C}_1,...,\hat{C}_5\}$, which are the maximal cliques of $\mathcal{G}$. Clients exceeding their communication threshold are highlighted in orange. Currently, client $\theta_6$ has exceeded the communication threshold $D_1$ for cluster $\hat{C}_1$, which causes cluster $\hat{C}_1$ to be added to the queue. The server pops cluster $\hat{C}_2$ from the queue and facilitates collaboration among $\{\theta_1, \theta_3, \theta_7\}$. In the next timestep, the server will serve cluster $\hat{C}_4$, queued for removal.
• Figure 2: Experimental Results on Simulated Dataset
• Figure 3: Experimental Results on LastFM Dataset
• Figure 4: Experimental Results on Imbalanced Synthetic Dataset

Theorems & Definitions (16)

• Theorem 1: Clustering Correctness
• Lemma 2: Confidence Ellipsoids
• Theorem 3: Regret and Communication Cost
• Remark 4
• Lemma 5: Lemma 11 in abbasi2011improved
• Lemma 6: Theorem 1 of abbasi2011improved
• Lemma 7: Determinant-Trace Inequality
• Lemma 8: Lemma 12 from lidyclu
• Lemma 9: Lemma B1 from Liasynch
• Lemma 10