Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DIML: Differentiable Inverse Mechanism Learning from Behaviors of Multi-Agent Learning Trajectories

Zhiyu An, Wan Du

TL;DR

The paper tackles the inverse problem of recovering an unknown incentive mechanism from observed multi-agent learning trajectories, addressing the gap left by equilibrium-focused inverse methods and forward differentiable mechanism design. It introduces DIML, a likelihood-based framework that differentiates through learning dynamics by evaluating counterfactual payoffs under a candidate mechanism, enabling scalable reconstruction of high-capacity, unobserved mechanisms $M_ heta:igotimes_i\mathcal{A}_i\to\mathbb{R}^n$ from action traces. It establishes identifiability of payoff differences under a conditional logit response and proves the statistical consistency of maximum likelihood estimation within identifiable parametric classes, while evaluating DIML across neural, structured, and large-scale environments with open-source code. The results demonstrate DIML’s ability to recover incentive differences and support reliable counterfactual prediction, scaling to hundreds of participants and offering a practical tool for auditing and counterfactual analysis in complex incentive systems.

Abstract

We study inverse mechanism learning: recovering an unknown incentive-generating mechanism from observed strategic interaction traces of self-interested learning agents. Unlike inverse game theory and multi-agent inverse reinforcement learning, which typically infer utility/reward parameters inside a structured mechanism, our target includes unstructured mechanism -- a (possibly neural) mapping from joint actions to per-agent payoffs. Unlike differentiable mechanism design, which optimizes mechanisms forward, we infer mechanisms from behavior in an observational setting. We propose DIML, a likelihood-based framework that differentiates through a model of multi-agent learning dynamics and uses the candidate mechanism to generate counterfactual payoffs needed to predict observed actions. We establish identifiability of payoff differences under a conditional logit response model and prove statistical consistency of maximum likelihood estimation under standard regularity conditions. We evaluate DIML with simulated interactions of learning agents across unstructured neural mechanisms, congestion tolling, public goods subsidies, and large-scale anonymous games. DIML reliably recovers identifiable incentive differences and supports counterfactual prediction, where its performance rivals tabular enumeration oracle in small environments and its convergence scales to large, hundred-participant environments. Code to reproduce our experiments is open-sourced.

DIML: Differentiable Inverse Mechanism Learning from Behaviors of Multi-Agent Learning Trajectories

TL;DR

The paper tackles the inverse problem of recovering an unknown incentive mechanism from observed multi-agent learning trajectories, addressing the gap left by equilibrium-focused inverse methods and forward differentiable mechanism design. It introduces DIML, a likelihood-based framework that differentiates through learning dynamics by evaluating counterfactual payoffs under a candidate mechanism, enabling scalable reconstruction of high-capacity, unobserved mechanisms from action traces. It establishes identifiability of payoff differences under a conditional logit response and proves the statistical consistency of maximum likelihood estimation within identifiable parametric classes, while evaluating DIML across neural, structured, and large-scale environments with open-source code. The results demonstrate DIML’s ability to recover incentive differences and support reliable counterfactual prediction, scaling to hundreds of participants and offering a practical tool for auditing and counterfactual analysis in complex incentive systems.

Abstract

We study inverse mechanism learning: recovering an unknown incentive-generating mechanism from observed strategic interaction traces of self-interested learning agents. Unlike inverse game theory and multi-agent inverse reinforcement learning, which typically infer utility/reward parameters inside a structured mechanism, our target includes unstructured mechanism -- a (possibly neural) mapping from joint actions to per-agent payoffs. Unlike differentiable mechanism design, which optimizes mechanisms forward, we infer mechanisms from behavior in an observational setting. We propose DIML, a likelihood-based framework that differentiates through a model of multi-agent learning dynamics and uses the candidate mechanism to generate counterfactual payoffs needed to predict observed actions. We establish identifiability of payoff differences under a conditional logit response model and prove statistical consistency of maximum likelihood estimation under standard regularity conditions. We evaluate DIML with simulated interactions of learning agents across unstructured neural mechanisms, congestion tolling, public goods subsidies, and large-scale anonymous games. DIML reliably recovers identifiable incentive differences and supports counterfactual prediction, where its performance rivals tabular enumeration oracle in small environments and its convergence scales to large, hundred-participant environments. Code to reproduce our experiments is open-sourced.
Paper Structure (35 sections, 3 theorems, 16 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 35 sections, 3 theorems, 16 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Core challenges.
  3. Our approach.
  4. Contributions.
  5. Related Work
  6. Inverse game theory and inverse equilibrium
  7. Multi-agent IRL, reward ambiguity, and solution concepts
  8. Differentiable mechanism design and bilevel incentive optimization
  9. Problem Setup
  10. Agent learning model and induced trajectory likelihood
  11. Inverse mechanism learning objective
  12. DIML: Differentiable Inverse Mechanism Learning
  13. From mechanisms to counterfactual payoff tensors
  14. Why counterfactual evaluation is essential.
  15. Learning dynamics model
  16. ...and 20 more sections

Key Result

Theorem 1

Fix agent $i$ and opponent action $a_{-i}\in\mathcal{A}_{-i}$. Under Assumption assump:logit with known $\beta>0$, the conditional choice probabilities $\{\mathbb{P}_\theta(a_i=a\mid a_{-i})\}_{a\in\mathcal{A}_i}$ identify the payoff differences $u_i^\theta(a,a_{-i}) - u_i^\theta(a',a_{-i})$ for all

Figures (1)

  • Figure 1: Mechanism recovery and counterfactual validity. Left panel in each subfigure: payoff-difference error (diff_mse); right panel: counterfactual KL under learner-parameter shifts (cfkl_params).

Theorems & Definitions (7)

  • Theorem 1: Identifiability of payoff differences
  • proof
  • Corollary 1: Gauge fixing via normalization
  • proof
  • Theorem 2: Consistency of MLE (conditional logit)
  • proof
  • Remark 1: Unknown temperature