Learning with Limited Shared Information in Multi-agent Multi-armed Bandit

TL;DR

This work addresses learning in multi-agent multi-armed bandits under limited information sharing due to privacy concerns. It introduces LSI-MAMAB and the Balanced-ETC algorithm, achieving asymptotically optimal social regret of $R(T)=O\left(\sum_{i=2}^N \frac{\log T}{\Delta_i}\right)$ while ensuring each agent’s average regret remains near a constant as the number of agents grows, thanks to a carefully designed balance constraint. An incentive mechanism is proposed to guarantee individual rationality, compensating data providers and charging users for shared information; theoretical IR guarantees are established and supported by experiments showing effective collaboration and platform profitability with sufficient participation. The results demonstrate that selective information sharing can sustain fast, coordinated learning without forcing privacy-compromising data disclosure, with practical implications for federated and privacy-aware collaborative decision-making.

Abstract

Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years. However, existing studies do not consider the case where an agent may refuse to share all her information with others, e.g., when some of the data contains personal privacy. In this paper, we propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share, and propose the Balanced-ETC algorithm to help multiple agents collaborate efficiently with limited shared information. Our analysis shows that Balanced-ETC is asymptotically optimal and its average regret (on each agent) approaches a constant when there are sufficient agents involved. Moreover, to encourage agents to participate in this collaborative learning, an incentive mechanism is proposed to make sure each agent can benefit from the collaboration system. Finally, we present experimental results to validate our theoretical results.

Figures (4)

• Figure 1: Illustration of the LSI-MAMAB model. In each time step $t$, the agents choose arms sequentially. After agent $m$ pulls an arm $\pi_m(t)$, she then gets a random reward $X_{\pi_m(t), m}(t) \sim D_{\pi_2(t)}$ from the arm $\pi_m(t)$. If $\pi_m(t) \in A_2$, then she could broadcast the reward, so that other agents can use this information. Otherwise, agent $m$ keeps this reward information to herself. The central controller wants to design an algorithm to minimize the overall regret. Besides, he also uses an incentive mechanism to achieve IR: after an agent pulls an arm, he provides some compensation to her, and charges other agents for the shared information (if the arm-reward pair is shared).
• Figure 2: Experimental results of Balanced-ETC
• Figure 3: regret with T
• Figure 4: experimental results of random setting

Theorems & Definitions (5)

• Theorem 4.1
• Theorem 4.7
• Lemma 5.1
• Theorem 5.2
• Lemma H.1