Personalized Dynamic Difficulty Adjustment -- Imitation Learning Meets Reinforcement Learning

TL;DR

Dynamic Difficulty Adjustment often fails when it cannot adapt to individual players. This paper introduces PDDA, a two-stage framework that couples an imitation-learning agent (to model player behavior) with a reinforcement-learning opponent (to beat the imitator), enabling real-time personalization through periodic opponent replacement. The approach is instantiated with River for fast imitation learning and an Advantage Actor Critic policy for RL, evaluated in FightingICE with a small user study showing higher user satisfaction than a baseline. The work demonstrates a minimal-design, plug-in method for personalized game challenge, with potential to generalize to more complex games and human players.

Abstract

Balancing game difficulty in video games is a key task to create interesting gaming experiences for players. Mismatching the game difficulty and a player's skill or commitment results in frustration or boredom on the player's side, and hence reduces time spent playing the game. In this work, we explore balancing game difficulty using machine learning-based agents to challenge players based on their current behavior. This is achieved by a combination of two agents, in which one learns to imitate the player, while the second is trained to beat the first. In our demo, we investigate the proposed framework for personalized dynamic difficulty adjustment of AI agents in the context of the fighting game AI competition.

Figures (1)

• Figure 1: Proposed agent architecture for personalized dynamic difficulty adjustment.