Mini Honor of Kings: A Lightweight Environment for Multi-Agent Reinforcement Learning

TL;DR

The paper addresses the need for lightweight, customizable MARL benchmarks by introducing Mini Honor of Kings (Mini HoK) built on a public Honor of Kings map editor. It presents a Python-accessible environment with a highly optimized simulator that supports curriculum and transfer learning, while remaining accessible on consumer hardware. Experimental results show classical MARL methods lag behind a heuristic baseline, underscoring room for methodological improvements and the value of diverse, heterogeneous team configurations. By open-sourcing the tool and providing multiple task modes, the work lowers barriers to MARL research and invites community-driven map design and algorithm development.

Abstract

Games are widely used as research environments for multi-agent reinforcement learning (MARL), but they pose three significant challenges: limited customization, high computational demands, and oversimplification. To address these issues, we introduce the first publicly available map editor for the popular mobile game Honor of Kings and design a lightweight environment, Mini Honor of Kings (Mini HoK), for researchers to conduct experiments. Mini HoK is highly efficient, allowing experiments to be run on personal PCs or laptops while still presenting sufficient challenges for existing MARL algorithms. We have tested our environment on common MARL algorithms and demonstrated that these algorithms have yet to find optimal solutions within this environment. This facilitates the dissemination and advancement of MARL methods within the research community. Additionally, we hope that more researchers will leverage the Honor of Kings map editor to develop innovative and scientifically valuable new maps. Our code and user manual are available at: https://github.com/tencent-ailab/mini-hok.

Figures (14)

• Figure 1: Examples to show characteristics of our environment. (a) An example to show the battling scenario of our Mini HoK environment, where heroes are cooperating to fight against the Dark Dragon. (b) An example to present the start positions of heroes in the environment. The default start positions are designated as points 1-5. The middle point is initial position of the Dark Dragon.
• Figure 2: Examples to show the expansibility of our Mini Honor of Kings Environment. In figure (a) and (b), five heroes engage in battle against dragon, featuring Zhuang Zhou in the former and Zhang Fei in the latter. Figure (c) showcases a scenario with three heroes participating in combat. Also, the levels of the heroes and the dragon's health points can be adjusted. Additionally, users can toggle the presence of equipment for the heroes or other parameters within our environment, thereby offering a comprehensive array of customization options.
• Figure 3: Python example and config example.
• Figure 4: Experiment results for different settings that reflects the damage that the system can cause during the training. The solid lines show the average performance across 5 random seeds and the shaded areas reflect the standard error of the performance outcomes.
• Figure 5: The lazy agent scenario.