Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Inverse Multiagent Learning

Denizalp Goktas, Amy Greenwald, Sadie Zhao, Alec Koppel, Sumitra Ganesh

TL;DR

This work tackles inverse game-theoretic problems by formulating parameter recovery as zero-sum min-max optimization, enabling efficient computation of inverse Nash equilibria under suitable concavity and smoothness conditions. It extends the framework to inverse multiagent reinforcement learning with stochastic access and develops gradient-based methods (e.g., SGDA) that converge to inverse equilibria in polynomial time. The authors further generalize to simulacral learning, incorporating faithful and unfaithful observations via a min-max objective that couples observation loss with regret, and prove convergence properties through Moreau envelope constructs. Empirical results demonstrate the approach can recover market parameters in synthetic settings and outperform ARIMA in predicting Spanish electricity prices, highlighting practical impact for forecasting and mechanism design in multiagent systems.

Abstract

In this paper, we study inverse game theory (resp. inverse multiagent learning) in which the goal is to find parameters of a game's payoff functions for which the expected (resp. sampled) behavior is an equilibrium. We formulate these problems as generative-adversarial (i.e., min-max) optimization problems, for which we develop polynomial-time algorithms to solve, the former of which relies on an exact first-order oracle, and the latter, a stochastic one. We extend our approach to solve inverse multiagent simulacral learning in polynomial time and number of samples. In these problems, we seek a simulacrum, meaning parameters and an associated equilibrium that replicate the given observations in expectation. We find that our approach outperforms the widely-used ARIMA method in predicting prices in Spanish electricity markets based on time-series data.

Efficient Inverse Multiagent Learning

TL;DR

This work tackles inverse game-theoretic problems by formulating parameter recovery as zero-sum min-max optimization, enabling efficient computation of inverse Nash equilibria under suitable concavity and smoothness conditions. It extends the framework to inverse multiagent reinforcement learning with stochastic access and develops gradient-based methods (e.g., SGDA) that converge to inverse equilibria in polynomial time. The authors further generalize to simulacral learning, incorporating faithful and unfaithful observations via a min-max objective that couples observation loss with regret, and prove convergence properties through Moreau envelope constructs. Empirical results demonstrate the approach can recover market parameters in synthetic settings and outperform ARIMA in predicting Spanish electricity prices, highlighting practical impact for forecasting and mechanism design in multiagent systems.

Abstract

In this paper, we study inverse game theory (resp. inverse multiagent learning) in which the goal is to find parameters of a game's payoff functions for which the expected (resp. sampled) behavior is an equilibrium. We formulate these problems as generative-adversarial (i.e., min-max) optimization problems, for which we develop polynomial-time algorithms to solve, the former of which relies on an exact first-order oracle, and the latter, a stochastic one. We extend our approach to solve inverse multiagent simulacral learning in polynomial time and number of samples. In these problems, we seek a simulacrum, meaning parameters and an associated equilibrium that replicate the given observations in expectation. We find that our approach outperforms the widely-used ARIMA method in predicting prices in Spanish electricity markets based on time-series data.

Paper Structure

This paper contains 27 sections, 11 theorems, 26 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preliminaries
  4. Inverse Multiagent Planning
  5. Inverse Multiagent Reinforcement Learning
  6. Simulacral Learning
  7. Experiments
  8. Appendix
  9. Additional Preliminary Definitions
  10. Notation
  11. Ommited Proofs
  12. Related Work
  13. Microeconomics
  14. Econometrics
  15. Inverse Optimization.
  16. ...and 12 more sections

Key Result

Theorem 3.1

The set of inverse NE of $\game[][-1]$ is the set of parameter profiles $\param \in \Theta$ that solve the optimization problem $\min_{\substack{\param \in \Theta}} \exploit (\truestrat; \param)$, or equivalently, this min-max optimization problem:

Figures (4)

  • Figure 1: Hourly prices in the Spanish electricity market from January 2015 to December 2020. The Nash simulacrum achieves a MSE that is twice as low as that of the ARIMA method.
  • Figure : Adversarial Inverse Multiagent Planning
  • Figure : Adversarial Inverse MARL
  • Figure : Adversarial Simulacral Learning

Theorems & Definitions (19)

  • Theorem 3.1
  • Remark 1
  • Theorem 3.2: Inverse NE Complexity
  • Corollary 1
  • Theorem 4.1
  • Theorem 5.1
  • Theorem 5.2
  • Theorem 7.1
  • proof : Proof of \ref{['thm:inverse_NE']}
  • proof : Proof of \ref{['thm:inverse_NE']}
  • ...and 9 more