ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

TL;DR

ELF introduces an extensive, lightweight, and flexible RL platform tailored for real-time strategy (RTS) research, featuring three environments (Mini-RTS, Capture the Flag, Tower Defense) and a fast, concurrent C++/Python simulation backend. The paper demonstrates end-to-end training of full-game RTS bots using A3C, with architectural choices like Leaky ReLU and Batch Normalization, long-horizon training, and curriculum learning yielding strong performance against built-in AIs. It also benchmarks throughput, compares baselines including MCTS, and discusses forward-planning integrations, highlighting the platform’s scalability and research utility. ELF is open-sourced to enable rapid exploration of hierarchical RL, planning under uncertainty, and multi-topology training in RTS settings.

Abstract

In this paper, we propose ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research. Using ELF, we implement a highly customizable real-time strategy (RTS) engine with three game environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a miniature version of StarCraft, captures key game dynamics and runs at 40K frame-per-second (FPS) per core on a Macbook Pro notebook. When coupled with modern reinforcement learning methods, the system can train a full-game bot against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition, our platform is flexible in terms of environment-agent communication topologies, choices of RL methods, changes in game parameters, and can host existing C/C++-based game environments like Arcade Learning Environment. Using ELF, we thoroughly explore training parameters and show that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than $70\%$ of the time in the full game of Mini-RTS. Strong performance is also achieved on the other two games. In game replays, we show our agents learn interesting strategies. ELF, along with its RL platform, is open-sourced at https://github.com/facebookresearch/ELF.

Figures (8)

• Figure 1: Overview of ELF.
• Figure 2: Hierarchical layout of ELF. In the current repository (https://github.com/facebookresearch/ELF, master branch), there are board games (e.g., Go darkforest), Atari learning environment ale, and a customized RTS engine that contains three simple games.
• Figure 3: Overview of Real-time strategy engine. (a) Visualization of current game state. (b) The three different game environments and their descriptions.
• Figure 4: Frame-per-second per CPU core (no hyper-threading) with respect to CPUs/threads. ELF (light-shaded) is 3x faster than OpenAI Gym openai-gym (dark-shaded) with 1024 threads. CPU involved in testing: Intel E5-2680@2.50GHz.
• Figure 5: Win rate in Mini-RTS with respect to the amount of experience at different steps $T$ in A3C. Note that one sample (with history) in $T = 2$ is equivalent to two samples in $T = 1$. Longer $T$ shows superior performance to small step counterparts, even if their samples are more expensive.