Responding to Promises: No-regret learning against followers with memory
Vijeth Hebbar, Cédric Langbort
TL;DR
The paper investigates learning optimal leader strategies in repeated Stackelberg games where followers are unknown, memory-enabled, and may exhibit bounded rationality via a quantal response. By formulating the problem within online learning and employing Follow-The-Perturbed-Leader (FTPL) with carefully designed perturbations, the authors prove sublinear regret against memoryless QR followers and against followers with bounded memory, under full-information feedback. The key technical insight is leveraging the QR model's smoothness to bound stability terms in the regret analysis, enabling no-regret guarantees even when followers base decisions on an aggregate of past leader plays. Computationally, the work discusses oracle constructions and acknowledges the NP-hardness of exact BR-based solutions, suggesting practical non-convex solvers for QR and MIP-based approaches for BR, with simulations validating the theoretical bounds. This contributes a robust framework for analyzing reputation-considerate leadership in hierarchical, learning-enabled environments with boundedly rational agents, with implications for security, economics, and automated decision-making under uncertainty about follower behavior.
Abstract
We consider a repeated Stackelberg game setup where the leader faces a sequence of followers of unknown types and must learn what commitments to make. While previous works have considered followers that best respond to the commitment announced by the leader in every round, we relax this setup in two ways. Motivated by natural scenarios where the leader's reputation factors into how the followers choose their response, we consider followers with memory. Specifically, we model followers that base their response on not just the leader's current commitment but on an aggregate of their past commitments. In developing learning strategies that the leader can employ against such followers, we make the second relaxation and assume boundedly rational followers. In particular, we focus on followers employing quantal responses. Interestingly, we observe that the smoothness property offered by the quantal response (QR) model helps in addressing the challenge posed by learning against followers with memory. Utilizing techniques from online learning, we develop algorithms that guarantee $O(\sqrt{T})$ regret for quantal responding memory-less followers and $O(\sqrt{BT})$ regret for followers with bounded memory of length $B$ with both scaling polynomially in game parameters.