Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

COMBAT: Conditional World Models for Behavioral Agent Training

Anmol Agarwal, Pranay Meshram, Sumer Singh, Saurav Suman, Andrew Lapp, Shahbuland Matiana, Louis Castricato, Spencer Frazier

TL;DR

This work demonstrates that diffusion models can successfully simulate a dynamic opponent that reacts to player actions, learning its behavior implicitly, and observes the emergence of sophisticated agent behavior by training the model solely on single-player inputs, without any explicit supervision for the opponent's policy.

Abstract

Recent advances in video generation have spurred the development of world models capable of simulating 3D-consistent environments and interactions with static objects. However, a significant limitation remains in their ability to model dynamic, reactive agents that can intelligently influence and interact with the world. To address this gap, we introduce COMBAT, a real-time, action-controlled world model trained on the complex 1v1 fighting game Tekken 3. Our work demonstrates that diffusion models can successfully simulate a dynamic opponent that reacts to player actions, learning its behavior implicitly. Our approach utilizes a 1.2 billion parameter Diffusion Transformer, conditioned on latent representations from a deep compression autoencoder. We employ state-of-the-art techniques, including causal distillation and diffusion forcing, to achieve real-time inference. Crucially, we observe the emergence of sophisticated agent behavior by training the model solely on single-player inputs, without any explicit supervision for the opponent's policy. Unlike traditional imitation learning methods, which require complete action labels, COMBAT learns effectively from partially observed data to generate responsive behaviors for a controllable Player 1. We present an extensive study and introduce novel evaluation methods to benchmark this emergent agent behavior, establishing a strong foundation for training interactive agents within diffusion-based world models.

COMBAT: Conditional World Models for Behavioral Agent Training

TL;DR

This work demonstrates that diffusion models can successfully simulate a dynamic opponent that reacts to player actions, learning its behavior implicitly, and observes the emergence of sophisticated agent behavior by training the model solely on single-player inputs, without any explicit supervision for the opponent's policy.

Abstract

Recent advances in video generation have spurred the development of world models capable of simulating 3D-consistent environments and interactions with static objects. However, a significant limitation remains in their ability to model dynamic, reactive agents that can intelligently influence and interact with the world. To address this gap, we introduce COMBAT, a real-time, action-controlled world model trained on the complex 1v1 fighting game Tekken 3. Our work demonstrates that diffusion models can successfully simulate a dynamic opponent that reacts to player actions, learning its behavior implicitly. Our approach utilizes a 1.2 billion parameter Diffusion Transformer, conditioned on latent representations from a deep compression autoencoder. We employ state-of-the-art techniques, including causal distillation and diffusion forcing, to achieve real-time inference. Crucially, we observe the emergence of sophisticated agent behavior by training the model solely on single-player inputs, without any explicit supervision for the opponent's policy. Unlike traditional imitation learning methods, which require complete action labels, COMBAT learns effectively from partially observed data to generate responsive behaviors for a controllable Player 1. We present an extensive study and introduce novel evaluation methods to benchmark this emergent agent behavior, establishing a strong foundation for training interactive agents within diffusion-based world models.
Paper Structure (24 sections, 6 equations, 5 figures, 2 tables)

This paper contains 24 sections, 6 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Video Diffusion Models
  4. Neural Game Engines and World Models
  5. Multi-Modal and Behavioral World Models
  6. Optimization Techniques for Interactive Generation
  7. Method
  8. Problem Formulation
  9. Tekken 3 Gameplay Dataset
  10. Model Architecture
  11. Multi-Modal Latent Encoding
  12. Diffusion Transformer Backbone
  13. Accelerated Inference for Real-Time Generation
  14. Experiments
  15. Implementation Details
  16. ...and 9 more sections

Figures (5)

  • Figure 1: An overview of the COMBAT world model. (Top) The model is conditioned on the current state (visual frames and poses) and Player 1's control inputs to autoregressively predict subsequent frames. (Bottom) Three distinct generated trajectories showcase the model's ability to produce plausible, strategic counter-attacks from Player 2 as an emergent response to Player 1's actions, without direct supervision of the opponent's policy.
  • Figure 2: Architectural diagram of the COMBAT model. (a) The end-to-end training process, where a Diffusion Transformer is conditioned on action and timestep embeddings to denoise latent frame representations. (b) The internal structure of the DiT backbone, which employs a hybrid local-global attention pattern to efficiently model long-term dependencies.
  • Figure 3: Behavioral Consistency Metrics. A comparison of generated gameplay (COMBAT) against the ground truth. (a, b) The per-frame damage distributions for Player 1 and Player 2, showing that our model learns a realistic mapping of actions to consequences. (c, d) The mean health trajectories over the course of a round, indicating that COMBAT captures the natural pacing of a match.
  • Figure 4: Total Action Adherence across training checkpoints
  • Figure 5: Action Ratio Consistency across training checkpoints