RRLS : Robust Reinforcement Learning Suite
Adil Zouitine, David Bertoin, Pierre Clavier, Matthieu Geist, Emmanuel Rachelson
TL;DR
This work addresses the lack of standardized benchmarks for robust reinforcement learning by introducing RRLS, a Mujoco-based benchmark suite with six continuous-control tasks and two uncertainty sets. It formalizes robust RL within the robust MDP framework and adopts a two-player deep robust RL paradigm to evaluate policies against adversarial transition dynamics. RRLS provides wrappers and evaluation procedures to enable reproducible comparisons across methods, and demonstrates baseline performance (TD3, Domain Randomization, NR-MDP, RARL, M2TD3) showing trade-offs between worst-case robustness and average-case performance. The benchmark and accompanying code aim to standardize robust RL experiments, facilitating fair comparisons and accelerating progress toward deployable, robust controllers in real-world uncertain environments.
Abstract
Robust reinforcement learning is the problem of learning control policies that provide optimal worst-case performance against a span of adversarial environments. It is a crucial ingredient for deploying algorithms in real-world scenarios with prevalent environmental uncertainties and has been a long-standing object of attention in the community, without a standardized set of benchmarks. This contribution endeavors to fill this gap. We introduce the Robust Reinforcement Learning Suite (RRLS), a benchmark suite based on Mujoco environments. RRLS provides six continuous control tasks with two types of uncertainty sets for training and evaluation. Our benchmark aims to standardize robust reinforcement learning tasks, facilitating reproducible and comparable experiments, in particular those from recent state-of-the-art contributions, for which we demonstrate the use of RRLS. It is also designed to be easily expandable to new environments. The source code is available at \href{https://github.com/SuReLI/RRLS}{https://github.com/SuReLI/RRLS}.