Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Thompson sampling under constrained communication

Saba Zerefa, Zhaolin Ren, Haitong Ma, Na Li

TL;DR

This work addresses distributed Bayesian optimization with constrained communication by introducing a distributed Thompson sampling algorithm. Each of M agents maintains its own Gaussian process and communicates sampled points with neighbors on a graph, yielding regret bounds that depend on the graph’s structure, notably the clique cover number $\theta(G)$. Theoretical results show that average regret scales as $\tilde{O}\left(\frac{\sqrt{\theta(G)}}{\sqrt{M t}}\right)$ and simple regret benefits from larger complete subgraphs, while numerical experiments demonstrate faster convergence on more connected graphs. The approach enables scalable, data-efficient optimization in networks with limited communication, with practical impact for multi-robot, sensor, and distributed design optimization settings.

Abstract

In Bayesian optimization, a black-box function is maximized via the use of a surrogate model. We apply distributed Thompson sampling, using a Gaussian process as a surrogate model, to approach the multi-agent Bayesian optimization problem. In our distributed Thompson sampling implementation, each agent receives sampled points from neighbors, where the communication network is encoded in a graph; each agent utilizes their own Gaussian process to model the objective function. We demonstrate theoretical bounds on Bayesian average regret and Bayesian simple regret, where the bound depends on the structure of the communication graph. Unlike in batch Bayesian optimization, this bound is applicable in cases where the communication graph amongst agents is constrained. When compared to sequential single-agent Thompson sampling, our bound guarantees faster convergence with respect to time as long as the communication graph is connected. We confirm the efficacy of our algorithm with numerical simulations on traditional optimization test functions, demonstrating the significance of graph connectivity on improving regret convergence.

Distributed Thompson sampling under constrained communication

TL;DR

This work addresses distributed Bayesian optimization with constrained communication by introducing a distributed Thompson sampling algorithm. Each of M agents maintains its own Gaussian process and communicates sampled points with neighbors on a graph, yielding regret bounds that depend on the graph’s structure, notably the clique cover number . Theoretical results show that average regret scales as and simple regret benefits from larger complete subgraphs, while numerical experiments demonstrate faster convergence on more connected graphs. The approach enables scalable, data-efficient optimization in networks with limited communication, with practical impact for multi-robot, sensor, and distributed design optimization settings.

Abstract

In Bayesian optimization, a black-box function is maximized via the use of a surrogate model. We apply distributed Thompson sampling, using a Gaussian process as a surrogate model, to approach the multi-agent Bayesian optimization problem. In our distributed Thompson sampling implementation, each agent receives sampled points from neighbors, where the communication network is encoded in a graph; each agent utilizes their own Gaussian process to model the objective function. We demonstrate theoretical bounds on Bayesian average regret and Bayesian simple regret, where the bound depends on the structure of the communication graph. Unlike in batch Bayesian optimization, this bound is applicable in cases where the communication graph amongst agents is constrained. When compared to sequential single-agent Thompson sampling, our bound guarantees faster convergence with respect to time as long as the communication graph is connected. We confirm the efficacy of our algorithm with numerical simulations on traditional optimization test functions, demonstrating the significance of graph connectivity on improving regret convergence.

Paper Structure

This paper contains 15 sections, 9 theorems, 25 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation and Preliminaries
  3. Problem Formulation
  4. Gaussian Process
  5. Regret
  6. Thompson Sampling
  7. Algorithm: Distributed Thompson Sampling
  8. Theoretical Result
  9. Numerical Experiments
  10. Simulation
  11. Discussion
  12. Conclusion
  13. Additional Analysis
  14. Bounds for the information gain quantity $\Psi_{\tau}$ for different kernels
  15. Objective Functions

Key Result

Theorem 3.1

Suppose $k(x,x')\leq 1$ for all $x,x'.$ Let $\{G_k\}_{k\in\{1,\ldots,n\}}$ be a collection of $n$ disjoint complete subgraphs of communication graph $G=(V,E)$, where $G_k=(V_k,E_k),$ and $\cup_{k\in\{1,\ldots,n\}}V_k=V.$ Then the Bayesian average regret after $t$ timesteps satisfies $R_{AB}(t)\leq \

Figures (2)

  • Figure 1: Regret analysis of numerical simulations with $20$ agents on Erdős-Rényi random graphs with connectivity probability $0.2$ (blue), $0.4$ (orange), and $0.6$ (green), on Rosenbrock (left) and Ackley (right) objective functions.
  • Figure 2: Plots of test objective functions.

Theorems & Definitions (9)

  • Theorem 3.1
  • Corollary 3.2
  • Theorem 3.3
  • Corollary 3.4
  • Lemma I.1
  • Lemma I.2
  • Lemma I.3
  • Lemma I.4
  • Lemma II.1: cf. Theorem 5 in kakade