Learning the Expected Core of Strictly Convex Stochastic Cooperative Games

TL;DR

This paper considers the core learning problem in stochastic cooperative games, where the reward distribution is unknown, and presents an algorithm named Common-Points-Picking that returns a point in the expected core given a polynomial number of samples, with high probability.

Abstract

Reward allocation, also known as the credit assignment problem, has been an important topic in economics, engineering, and machine learning. An important concept in reward allocation is the core, which is the set of stable allocations where no agent has the motivation to deviate from the grand coalition. In previous works, computing the core requires either knowledge of the reward function in deterministic games or the reward distribution in stochastic games. However, this is unrealistic, as the reward function or distribution is often only partially known and may be subject to uncertainty. In this paper, we consider the core learning problem in stochastic cooperative games, where the reward distribution is unknown. Our goal is to learn the expected core, that is, the set of allocations that are stable in expectation, given an oracle that returns a stochastic reward for an enquired coalition each round. Within the class of strictly convex games, we present an algorithm named \texttt{Common-Points-Picking} that returns a point in the expected core given a polynomial number of samples, with high probability. To analyse the algorithm, we develop a new extension of the separation hyperplane theorem for multiple convex sets.

Figures (5)

• Figure 1: Set of common points constructed by separating hyperplanes. The intersection of half-spaces defined by $H_p(Q), \; \forall p\in [n],$ creates a subset of common points. The common points are in $\text{E-Core}$, provided that the confidence sets contain the ground-truth vertices.
• Figure 2: Simulation with game of $n \in \{2,...,10\}$ players, where the strict convexity constant $\varsigma$ is $0.1/n$ in the LHS and $0$ in the RHS.
• Figure 3: $c_W$ with $n\in\{10,\; 50,\; 100,\; 150,\; 200,\; 300,\; 500,\; 1000\}$, $\varsigma = \frac{0.1}{n}$, and $20000$ trials
• Figure : Common Points Picking
• Figure : Stopping Condition

Theorems & Definitions (38)

• Definition 1: Convex stochastic cooperative game
• Definition 2: $\varsigma$-Strictly convex cooperative game
• Definition 3: Expected core Pantazis2023_ExpectedCore
• Proposition 5
• Remark 6
• Theorem 7
• Example 8
• Proposition 9
• Definition 10: Separating hyperplane
• Theorem 11: Hyperplane separation theorem for multiple convex sets