Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ORACLE: Orchestrate NPC Daily Activities using Contrastive Learning with Transformer-CVAE

Seong-Eun Hong, JuYeong Hwang, RyunHa Lee, HyeongYeop Kang

Abstract

The integration of Non-player characters (NPCs) within digital environments has been increasingly recognized for its potential to augment user immersion and cognitive engagement. The sophisticated orchestration of their daily activities, reflecting the nuances of human daily routines, contributes significantly to the realism of digital environments. Nevertheless, conventional approaches often produce monotonous repetition, falling short of capturing the intricacies of real human activity plans. In response to this, we introduce ORACLE, a novel generative model for the synthesis of realistic indoor daily activity plans, ensuring NPCs' authentic presence in digital habitats. Exploiting the CASAS smart home dataset's 24-hour indoor activity sequences, ORACLE addresses challenges in the dataset, including its imbalanced sequential data, the scarcity of training samples, and the absence of pre-trained models encapsulating human daily activity patterns. ORACLE's training leverages the sequential data processing prowess of Transformers, the generative controllability of Conditional Variational Autoencoders (CVAE), and the discriminative refinement of contrastive learning. Our experimental results validate the superiority of generating NPC activity plans and the efficacy of our design strategies over existing methods.

ORACLE: Orchestrate NPC Daily Activities using Contrastive Learning with Transformer-CVAE

Abstract

The integration of Non-player characters (NPCs) within digital environments has been increasingly recognized for its potential to augment user immersion and cognitive engagement. The sophisticated orchestration of their daily activities, reflecting the nuances of human daily routines, contributes significantly to the realism of digital environments. Nevertheless, conventional approaches often produce monotonous repetition, falling short of capturing the intricacies of real human activity plans. In response to this, we introduce ORACLE, a novel generative model for the synthesis of realistic indoor daily activity plans, ensuring NPCs' authentic presence in digital habitats. Exploiting the CASAS smart home dataset's 24-hour indoor activity sequences, ORACLE addresses challenges in the dataset, including its imbalanced sequential data, the scarcity of training samples, and the absence of pre-trained models encapsulating human daily activity patterns. ORACLE's training leverages the sequential data processing prowess of Transformers, the generative controllability of Conditional Variational Autoencoders (CVAE), and the discriminative refinement of contrastive learning. Our experimental results validate the superiority of generating NPC activity plans and the efficacy of our design strategies over existing methods.
Paper Structure (19 sections, 9 equations, 7 figures, 7 tables)

This paper contains 19 sections, 9 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Plausible NPC Activity Planning
  4. Sequential data analysis
  5. Dataset
  6. Preprocessing
  7. Data Filtering
  8. Methodology
  9. Transformer-CVAE model
  10. Contrastive Learning
  11. Experiments
  12. Implementation details
  13. Comparative Evaluation
  14. Metrics
  15. Evaluation Results
  16. ...and 4 more sections

Figures (7)

  • Figure 1: The majority of users responded that the proposed filtering criteria for each activity are suitable for use as the minimum conditions to evaluate the activity's plausibility.
  • Figure 2: The workflow of ORACLE framework.
  • Figure 3: (a) Unity virtual environment where an NPC executes activities. (b) Samples of the visualized plan.
  • Figure 4: User study evaluation results. (a) Random generation scenario results in Apartment and Home datasets. (b) Conditional generation scenario results in Apartment and Home datasets. Real data is sampled from the test dataset.
  • Figure 5: (a) Latent visualization of ORACLE and ORACLE -C. (b) Attention map of ORACLE and ORACLE -C.
  • ...and 2 more figures