Active Inverse Methods in Stackelberg Games with Bounded Rationality

TL;DR

The paper addresses inferring an unknown follower cost parameter in Stackelberg games where the follower behaves with bounded rationality. It develops an active inverse framework comprising (i) a Fisher-information based active learning method that achieves consistency and asymptotic normality, and (ii) a cost-aware active inverse game that balances exploration and exploitation and proves convergence to Stackelberg equilibrium in quadratic settings, aided by parameter consistency. The theoretical results are complemented by simulations showing faster and more accurate identification and equilibrium attainment than passive or random strategies. The work highlights the benefit of actively shaping game outcomes to improve learning efficiency in strategic interactions with boundedly rational agents. The approach has potential impact on control, economics, and robotics where leader-follower dynamics and human-like rationality are present.

Abstract

Inverse game theory is utilized to infer the cost functions of all players based on game outcomes. However, existing inverse game theory methods do not consider the learner as an active participant in the game, which could significantly enhance the learning process. In this paper, we extend inverse game theory to active inverse methods. For Stackelberg games with bounded rationality, the leader, acting as a learner, actively chooses actions to better understand the follower's cost functions. First, we develop a method of active learning by leveraging Fisher information to maximize information gain about the unknown parameters and prove the consistency and asymptotic normality. Additionally, when leaders consider its cost, we develop a method of active inverse game to balance exploration and exploitation, and prove the consistency and asymptotic Stackelberg equilibrium with quadratic cost functions. Finally, we verify the properties of these methods through simulations in the quadratic case and demonstrate that the active inverse game method can achieve Stackelberg equilibrium more quickly through active exploration.

Figures (4)

• Figure 1: The relationship develops from IOC/IRL to inverse game theory and active inverse methods.
• Figure 2: Active learning for Stackelberg games with D-optimality, A-optimality and E-optimality vs. uniform random strategy.
• Figure 3: The relative error $\frac{\|\mathbf{u}^L(T) - \mathbf{u}^{L*}\|}{\|\mathbf{u}^{L*}\|}$ of the active inverse game for Stackelberg games (Algorithm 2) vs. strategy without exploration. The dotted lines represent the best results (Min), and the solid line represents the median of the results (Median) of 300 experiments by methods, respectively.
• Figure 4: Comparison of parameter estimate by active inverse methods in Stackelberg games and strategy without exploration by minimizing estimated expected cost over time at steps 20, 40, 60, 80 and 100.

Theorems & Definitions (15)

• Definition 1
• Remark 1
• Proposition 1
• Theorem 1: Consistency of Alg. \ref{['alg:fisher']}
• Theorem 2
• Corollary 1
• Lemma 1: Theorem 2 in hoadley1971asymptotic
• Theorem 3: Asymptotic normality of Alg. \ref{['alg:fisher']}
• Remark 2
• Remark 3