FDQN: A Flexible Deep Q-Network Framework for Game Automation

TL;DR

FDQN addresses real-time, high-dimensional decision making in dynamic environments, including HTML-based games such as Chrome Dino and a suite of Atari titles. It combines a CNN-based state encoder with a modular DQN architecture that adapts to varying action spaces and uses an epsilon-greedy policy along with a large replay buffer to stabilize learning. Empirical results show FDQN outperforming baseline DQN and Double DQN approaches across multiple games, indicating improved sample efficiency and robustness for online play. The work suggests broader applicability of flexible, scalable DQN designs to real-world, high-dimensional control tasks beyond gaming.

Abstract

In reinforcement learning, it is often difficult to automate high-dimensional, rapid decision-making in dynamic environments, especially when domains require real-time online interaction and adaptive strategies such as web-based games. This work proposes a state-of-the-art Flexible Deep Q-Network (FDQN) framework that can address this challenge with a selfadaptive approach that is processing high-dimensional sensory data in realtime using a CNN and dynamically adapting the model architecture to varying action spaces of different gaming environments and outperforming previous baseline models in various Atari games and the Chrome Dino game as baselines. Using the epsilon-greedy policy, it effectively balances the new learning and exploitation for improved performance, and it has been designed with a modular structure that it can be easily adapted to other HTML-based games without touching the core part of the framework. It is demonstrated that the FDQN framework can successfully solve a well-defined task in a laboratory condition, but more importantly it also discusses potential applications to more challenging real-world cases and serve as the starting point for future further exploration into automated game play and beyond.