Continual Depth-limited Responses for Computing Counter-strategies in Sequential Games

TL;DR

This work tackles exploiting imperfect opponent models in large sequential zero-sum games by merging limited look-ahead solving with explicit opponent modeling. It introduces continual depth-limited best response ($CDBR$) and continual depth-limited restricted Nash response ($CDRNR$), along with a full gadget to preserve soundness when using subgame solving. The authors provide theoretical guarantees on convergence, exploitability, and safety, and empirically demonstrate superior performance to baselines such as Local Best Response, Approximate BR, and SES across domains including Leduc Hold’em, Goofspiel, Liar’s Dice, and SlumBot in HUNL. The approach enables real-time computation of robust, exploitative strategies against unseen opponent models while maintaining controllable risk, with practical impact for AI agents in real-world imperfect-information games. The combination of rigorous theory and broad empirical validation highlights the potential of model-aware depth-limited solving for strategic decision making.

Abstract

In zero-sum games, the optimal strategy is well-defined by the Nash equilibrium. However, it is overly conservative when playing against suboptimal opponents and it can not exploit their weaknesses. Limited look-ahead game solving in imperfect-information games allows defeating human experts in massive real-world games such as Poker, Liar's Dice, and Scotland Yard. However, since they approximate Nash equilibrium, they tend to only win slightly against weak opponents. We propose methods combining limited look-ahead solving with an opponent model in order to 1) approximate a best response in large games or 2) compute a robust response with control over the robustness of the response. Both methods can compute the response in real time to previously unseen strategies. We present theoretical guarantees of our methods. We show that existing robust response methods do not work combined with limited look-ahead solving of the shelf, and we propose a novel solution for the issue. Our algorithm performs significantly better than multiple baselines in smaller games and outperforms state-of-the-art methods against SlumBot.

Figures (15)

• Figure 1: Illustration of the depth-limited solving.
• Figure 2: Simple zero-sum imperfect-information game. Nodes denote the decisions of the players, dotted lines mark information sets, and the leaf shows for player 1
• Figure 3: (left) A game to show problems with gadgets. (middle) Resolving gadget for the left game. (right) Max-margin and Reach max-margin gadget.
• Figure 4: Comparison of Gain and Exploitability of the solutions using Full gadget compared to other gadgets. Max-margin and resolving gadget lines are overlapping.
• Figure 5: Gain and exploitability comparison of BR, RNR, best Nash equilibrium (BNE), CDBR-NE, SES, and CDRNR in Leduc Hold'em against strategies from CFR using a small number of iterations with different $p$ values. The a stands for the average of the other values. VF is CDRNR using an imperfect value function.

Theorems & Definitions (9)

• lemma 1
• definition 1
• theorem 1: Gain of CDRNR
• theorem 2: Safety of CDRNR
• lemma 2
• theorem 1
• theorem 2
• theorem 3
• theorem 4