Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks

Yun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Lin Liu, Junfeng Yang, Lin Lai, Hongyang Qin, Minwen Deng, Juchao Zhuo, Deheng Ye, Qiang Fu, Wei Yang, Guang Yang, Lanxiao Huang, Xiangyang Ji

TL;DR

Hokoff is proposed, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research and reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.

Abstract

The advancement of Offline Reinforcement Learning (RL) and Offline Multi-Agent Reinforcement Learning (MARL) critically depends on the availability of high-quality, pre-collected offline datasets that represent real-world complexities and practical applications. However, existing datasets often fall short in their simplicity and lack of realism. To address this gap, we propose Hokoff, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research. This data is derived from Honor of Kings, a recognized Multiplayer Online Battle Arena (MOBA) game known for its intricate nature, closely resembling real-life situations. Utilizing this framework, we benchmark a variety of offline RL and offline MARL algorithms. We also introduce a novel baseline algorithm tailored for the inherent hierarchical action space of the game. We reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.

Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks

TL;DR

Hokoff is proposed, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research and reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.

Abstract

The advancement of Offline Reinforcement Learning (RL) and Offline Multi-Agent Reinforcement Learning (MARL) critically depends on the availability of high-quality, pre-collected offline datasets that represent real-world complexities and practical applications. However, existing datasets often fall short in their simplicity and lack of realism. To address this gap, we propose Hokoff, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research. This data is derived from Honor of Kings, a recognized Multiplayer Online Battle Arena (MOBA) game known for its intricate nature, closely resembling real-life situations. Utilizing this framework, we benchmark a variety of offline RL and offline MARL algorithms. We also introduce a novel baseline algorithm tailored for the inherent hierarchical action space of the game. We reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.
Paper Structure (37 sections, 4 equations, 5 figures, 18 tables)

This paper contains 37 sections, 4 equations, 5 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Offline RL and Offline MARL
  4. Offline Datasets
  5. Background
  6. Hokoff
  7. Framework
  8. Multi-Level Models
  9. Game Modes
  10. Honor of Kings Arena
  11. Honor of Kings 3v3 Arena
  12. Subtasks
  13. Datasets
  14. Datasets Details
  15. Benchmarking
  16. ...and 22 more sections

Figures (5)

  • Figure 1: (a) The Game replay user interface (UI) in HoK1v1. (b) The UI in HoK3v3. Important information and units of the game are highlighted using orange boxes.
  • Figure 2: The architecture of the framework. The sampling and evaluation modules should interact with the environment. Multi-Level Models are the foundation baseline models of these two modules, serving as opponents in the evaluation module and being on both sides in the sampling module, as described in Sec \ref{['sec:evaluation']}. The training module is responsible for training offline RL algorithms using fixed datasets and producing trained models for evaluation.
  • Figure 3: Violin diagrams of all datasets in HoK1v1.
  • Figure 4: Violin diagrams of all datasets in HoK3v3.
  • Figure 5: Action space in HoK1v1 wei2022honor