Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learning

András Antos, András Millinghoffer, Péter Antal

TL;DR

It is demonstrated that a new pure-exploration model, the semi-overlapping multi-(multi-armed) bandit (SOMMAB), in which a single evaluation provides distinct feedback to multiple bandits due to structural overlap among their arms, can be used to learn a support network from sparse candidate lists efficiently.

Abstract

Many modern AI and ML problems require evaluating partners' contributions through shared yet asymmetric, computationally intensive processes and the simultaneous selection of the most beneficial candidates. Sequential approaches to these problems can be unified under a new framework, Sequential Support Network Learning (SSNL), in which the goal is to select the most beneficial candidate set of partners for all participants using trials; that is, to learn a directed graph that represents the highest-performing contributions. We demonstrate that a new pure-exploration model, the semi-overlapping multi-(multi-armed) bandit (SOMMAB), in which a single evaluation provides distinct feedback to multiple bandits due to structural overlap among their arms, can be used to learn a support network from sparse candidate lists efficiently. We develop a generalized GapE algorithm for SOMMABs and derive new exponential error bounds that improve the best known constant in the exponent for multi-bandit best-arm identification. The bounds scale linearly with the degree of overlap, revealing significant sample-complexity gains arising from shared evaluations. From an application point of view, this work provides a theoretical foundation and improved performance guarantees for sequential learning tools for identifying support networks from sparse candidates in multiple learning problems, such as in multi-task learning (MTL), auxiliary task learning (ATL), federated learning (FL), and in multi-agent systems (MAS).

Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learning

TL;DR

It is demonstrated that a new pure-exploration model, the semi-overlapping multi-(multi-armed) bandit (SOMMAB), in which a single evaluation provides distinct feedback to multiple bandits due to structural overlap among their arms, can be used to learn a support network from sparse candidate lists efficiently.

Abstract

Many modern AI and ML problems require evaluating partners' contributions through shared yet asymmetric, computationally intensive processes and the simultaneous selection of the most beneficial candidates. Sequential approaches to these problems can be unified under a new framework, Sequential Support Network Learning (SSNL), in which the goal is to select the most beneficial candidate set of partners for all participants using trials; that is, to learn a directed graph that represents the highest-performing contributions. We demonstrate that a new pure-exploration model, the semi-overlapping multi-(multi-armed) bandit (SOMMAB), in which a single evaluation provides distinct feedback to multiple bandits due to structural overlap among their arms, can be used to learn a support network from sparse candidate lists efficiently. We develop a generalized GapE algorithm for SOMMABs and derive new exponential error bounds that improve the best known constant in the exponent for multi-bandit best-arm identification. The bounds scale linearly with the degree of overlap, revealing significant sample-complexity gains arising from shared evaluations. From an application point of view, this work provides a theoretical foundation and improved performance guarantees for sequential learning tools for identifying support networks from sparse candidates in multiple learning problems, such as in multi-task learning (MTL), auxiliary task learning (ATL), federated learning (FL), and in multi-agent systems (MAS).
Paper Structure (24 sections, 5 theorems, 58 equations, 2 figures, 1 table)

This paper contains 24 sections, 5 theorems, 58 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Duality and the support network learning problem
  3. Towards multi-bandits
  4. From duality to the semi-overlapping property
  5. Related work
  6. Multi-armed bandits and best-arm identification
  7. Multi-bandits, overlapping and semi-overlapping (structured) MMABs
  8. Combinatorial bandits and Monte Carlo Tree Search
  9. Structure learning and sparse candidate methods
  10. Computational efficiency of joint evaluations
  11. Task selection in multi-task and auxiliary task learning
  12. Client selection in federated learning
  13. Partner selection in multi-agent systems and multi-agent RL
  14. Dataset selection
  15. Multi-Bandit Problem Setup
  16. ...and 9 more sections

Key Result

Proposition 2

If we run GapE with parameters $l=1$ and $0<a \le 4\frac{n-MK}{9H}$, then its probability of error satisfies in particular for $a = 4\frac{n-MK}{9H}$, we have $\ell(n) \le 2MKn \exp\left(-\frac{n-MK}{144H}\right)$.

Figures (2)

  • Figure 1: Overview of the Semi-Overlapping Multi-bandit (SOMMAB) formulation under donor--recipient duality. (A)A priori candidate sets: for each entity $a,b,c,d$, we specify the initially admissible donor sets, which do not need to satisfy strong duality. In this example, the empty set is always included and the complete set is also a candidate for an entity. (B)Candidate sets under strong duality: enforcing complete entity duality and relational duality expands the candidate sets. In particular, if a set is admissible for one entity, then for each of its members, the corresponding role-swapped variant is also admissible, yielding a structurally consistent family of candidate sets across all entities. (C)SOMMAB: each entity induces a local MAB problem, where arms correspond to its admissible candidate donor sets. In a SOMMAB, arms may be semi-overlapping (SO) across entities, as different MABs may mutually share arms corresponding to role-swapped variants of candidate sets. The dual nature of SO arms is expressed using the plate notation: the cost of pulling a SO arm is a single cost for all corresponding MAB, whereas the random rewards are MAB-specific. Synchronized sequential learning proceeds in parallel across MABs for the entities, with trials corresponding to selecting and evaluating candidate donor sets. (D)Solution representation: once learning converges, each entity selects its best-performing donor set, resulting in a directed support network. Multi-node donor sets induce multiple incoming edges to a recipient, and sets of directed edges to a recipient represent learned contribution relationships. This network compactly summarizes the outcome of the SOMMAB learning process. Note that the support network may contain cycles, contribution scores can be asymmetric ($\alpha\neq \beta$), and contribution scores correspond to donor sets, as in the case of $\delta$ for the $\{b\rightarrow d, c\rightarrow d\}$ arc-connected edge set (or, with hypergraph representation, $\{b,c\}\rightarrow d$ hyperedge).
  • Figure 2: The pseudo-code of the generalized Gap-based Exploration (GapE(l)) algorithm.

Theorems & Definitions (10)

  • Definition 1
  • Proposition 2
  • Theorem 3
  • Remark 4: Choice of $l$
  • Example 1: Comparison
  • Remark 5
  • Remark 6: $M$-order semi-overlapping
  • Lemma 7
  • Lemma 8
  • Corollary 9