Inverse Mixed Strategy Games with Generative Trajectory Models

TL;DR

This work tackles the inverse game problem under uncertainty for multi-agent robotics by introducing an inverse mixed-strategy framework that leverages a CVAE-based generative trajectory model as the nominal mixed strategy and a differentiable BRNE solver. By learning the inter-agent cost $l_{\theta}$ and using online samples from the CVAE, the method infers Nash-optimal actions even with noisy observations and unknown objectives. Offline data informs the nominal strategy while online observations drive adaptation, yielding robust performance comparable to ground-truth and oracle baselines in simulated navigation with an unknown game model. The approach offers a practical path toward safe, real-time multi-agent coordination in human-robot scenarios, with potential extensions to more complex, heterogeneous agent settings.

Abstract

Game-theoretic models are effective tools for modeling multi-agent interactions, especially when robots need to coordinate with humans. However, applying these models requires inferring their specifications from observed behaviors -- a challenging task known as the inverse game problem. Existing inverse game approaches often struggle to account for behavioral uncertainty and measurement noise, and leverage both offline and online data. To address these limitations, we propose an inverse game method that integrates a generative trajectory model into a differentiable mixed-strategy game framework. By representing the mixed strategy with a conditional variational autoencoder (CVAE), our method can infer high-dimensional, multi-modal behavior distributions from noisy measurements while adapting in real-time to new observations. We extensively evaluate our method in a simulated navigation benchmark, where the observations are generated by an unknown game model. Despite the model mismatch, our method can infer Nash-optimal actions comparable to those of the ground-truth model and the oracle inverse game baseline, even in the presence of uncertain agent objectives and noisy measurements.

Figures (4)

• Figure 1: The pure strategy inverse game formula (left) often fails to handle uncertainty or infer from both offline and online data, while our mixed strategy formula (right) addresses these issues by inferring multi-modal distributions with a generative trajectory model.
• Figure 2: The Nash-optimal mixed strategies before (left) and after (right) solving the inverse game (\ref{['eq:full_inverse_mixed_mle']}). Solid black lines represent the demonstrated future states in the dataset.
• Figure 3: Snapshots from a test with our method (blue: robot).
• Figure 4: Quantitative results of the navigation benchmark. We show the median, quartiles, and distribution of the results. Our method has comparable performance with the ground-truth baseline and the oracle inverse game baseline, while not relying on privileged information (e.g., the iLQGames model and objective structure) and with noisy observations.

Theorems & Definitions (6)

• Definition 1: Pure strategy trajectory game
• Definition 2: Pure strategy Nash equilibrium
• Definition 3: Mixed strategy trajectory game
• Definition 4: Mixed strategy Nash equilibrium
• Definition 5: Inverse pure strategy game
• Definition 6: Inverse mixed strategy game