Robust Online Learning with Private Information
Kyohei Okumura
TL;DR
The paper addresses the vulnerability of standard online learning algorithms to strategic manipulation when learners hold private information, showing that no-ER and many no-WER methods can be exploited by adaptive environments. It introduces partial safety as a conservative robustness criterion and presents the Explore-Exploit-Punish (EEP) algorithm, which achieves no-WER while limiting exploitable leakage of private information, and a welfare-efficient variant ESEP that can improve total welfare under favorable conditions. The main contributions include impossibility results for traditional regret-based designs, a concrete partially safe no-WER algorithm, and a welfare-aware extension, highlighting a shift toward safeguards against manipulation in algorithm design. The findings have practical implications for deploying online learning in economic settings, such as auctions and pricing, where adverse selection and strategic behavior are prevalent, and motivate further study of robust, strategy-aware learning rules. Overall, the work advances a robust design framework that balances strong performance in well-behaved environments with protection against worst-case strategic exploitation.
Abstract
This paper investigates the robustness of online learning algorithms when learners possess private information. No-external-regret algorithms, prevalent in machine learning, are vulnerable to strategic manipulation, allowing an adaptive opponent to extract full surplus. Even standard no-weak-external-regret algorithms, designed for optimal learning in stationary environments, exhibit similar vulnerabilities. This raises a fundamental question: can a learner simultaneously prevent full surplus extraction by adaptive opponents while maintaining optimal performance in well-behaved environments? To address this, we model the problem as a two-player repeated game, where the learner with private information plays against the environment, facing ambiguity about the environment's types: stationary or adaptive. We introduce \emph{partial safety} as a key design criterion for online learning algorithms to prevent full surplus extraction. We then propose the \emph{Explore-Exploit-Punish} (\textsf{EEP}) algorithm and prove that it satisfies partial safety while achieving optimal learning in stationary environments, and has a variant that delivers improved welfare performance. Our findings highlight the risks of applying standard online learning algorithms in strategic settings with adverse selection. We advocate for a shift toward online learning algorithms that explicitly incorporate safeguards against strategic manipulation while ensuring strong learning performance.