Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning When to Ask: Simulation-Trained Humanoids for Mental-Health Diagnosis

Filippo Cenacchi, Deborah Richards, Longbing Cao

TL;DR

This work tackles the challenge of safely training humanoid interview agents for mental-health screening without real-world hardware wear. By simulating 276 high-fidelity MetaHuman patients in Unreal Engine 5 and optimizing a continuous control policy with uncertainty-aware, counterfactual replay, the authors compare TD3 against PPO and CEM and show substantial gains in interview completeness and social timing, while maintaining safety and robustness. Key contributions include a scalable agent-centered simulator, a safe learning loop that formalizes timing and rapport as controllable objectives, a thorough comparative study, and ablation analyses that illuminate the drivers of performance. The results suggest a viable sim-to-real path for clinician-supervised humanoid pilots, enabling rapid, repeatable exploration of probing strategies before in-clinic deployment.

Abstract

Testing humanoid robots with users is slow, causes wear, and limits iteration and diversity. Yet screening agents must master conversational timing, prosody, backchannels, and what to attend to in faces and speech for Depression and PTSD. Most simulators omit policy learning with nonverbal dynamics; many controllers chase task accuracy while underweighting trust, pacing, and rapport. We virtualise the humanoid as a conversational agent to train without hardware burden. Our agent-centred, simulation-first pipeline turns interview data into 276 Unreal Engine MetaHuman patients with synchronised speech, gaze/face, and head-torso poses, plus PHQ-8 and PCL-C flows. A perception-fusion-policy loop decides what and when to speak, when to backchannel, and how to avoid interruptions, under a safety shield. Training uses counterfactual replay (bounded nonverbal perturbations) and an uncertainty-aware turn manager that probes to reduce diagnostic ambiguity. Results are simulation-only; the humanoid is the transfer target. In comparing three controllers, a custom TD3 (Twin Delayed DDPG) outperformed PPO and CEM, achieving near-ceiling coverage with steadier pace at comparable rewards. Decision-quality analyses show negligible turn overlap, aligned cut timing, fewer clarification prompts, and shorter waits. Performance stays stable under modality dropout and a renderer swap, and rankings hold on a held-out patient split. Contributions: (1) an agent-centred simulator that turns interviews into 276 interactive patients with bounded nonverbal counterfactuals; (2) a safe learning loop that treats timing and rapport as first-class control variables; (3) a comparative study (TD3 vs PPO/CEM) with clear gains in completeness and social timing; and (4) ablations and robustness analyses explaining the gains and enabling clinician-supervised humanoid pilots.

Learning When to Ask: Simulation-Trained Humanoids for Mental-Health Diagnosis

TL;DR

This work tackles the challenge of safely training humanoid interview agents for mental-health screening without real-world hardware wear. By simulating 276 high-fidelity MetaHuman patients in Unreal Engine 5 and optimizing a continuous control policy with uncertainty-aware, counterfactual replay, the authors compare TD3 against PPO and CEM and show substantial gains in interview completeness and social timing, while maintaining safety and robustness. Key contributions include a scalable agent-centered simulator, a safe learning loop that formalizes timing and rapport as controllable objectives, a thorough comparative study, and ablation analyses that illuminate the drivers of performance. The results suggest a viable sim-to-real path for clinician-supervised humanoid pilots, enabling rapid, repeatable exploration of probing strategies before in-clinic deployment.

Abstract

Testing humanoid robots with users is slow, causes wear, and limits iteration and diversity. Yet screening agents must master conversational timing, prosody, backchannels, and what to attend to in faces and speech for Depression and PTSD. Most simulators omit policy learning with nonverbal dynamics; many controllers chase task accuracy while underweighting trust, pacing, and rapport. We virtualise the humanoid as a conversational agent to train without hardware burden. Our agent-centred, simulation-first pipeline turns interview data into 276 Unreal Engine MetaHuman patients with synchronised speech, gaze/face, and head-torso poses, plus PHQ-8 and PCL-C flows. A perception-fusion-policy loop decides what and when to speak, when to backchannel, and how to avoid interruptions, under a safety shield. Training uses counterfactual replay (bounded nonverbal perturbations) and an uncertainty-aware turn manager that probes to reduce diagnostic ambiguity. Results are simulation-only; the humanoid is the transfer target. In comparing three controllers, a custom TD3 (Twin Delayed DDPG) outperformed PPO and CEM, achieving near-ceiling coverage with steadier pace at comparable rewards. Decision-quality analyses show negligible turn overlap, aligned cut timing, fewer clarification prompts, and shorter waits. Performance stays stable under modality dropout and a renderer swap, and rankings hold on a held-out patient split. Contributions: (1) an agent-centred simulator that turns interviews into 276 interactive patients with bounded nonverbal counterfactuals; (2) a safe learning loop that treats timing and rapport as first-class control variables; (3) a comparative study (TD3 vs PPO/CEM) with clear gains in completeness and social timing; and (4) ablations and robustness analyses explaining the gains and enabling clinician-supervised humanoid pilots.

Paper Structure

This paper contains 34 sections, 7 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Simulation and High-Fidelity Avatars for Clinical HRI
  4. Multimodal Mental-Health Computing and Clinical Corpora
  5. Reinforcement Learning for Dialogue Probing and Trust-Aware Control
  6. System Overview
  7. Humanoid–Avatar Interaction Loop.
  8. MetaHuman Patient Runner.
  9. Policy Learning Stack.
  10. Trust- and Uncertainty-Aware Control.
  11. Safety and Auditability.
  12. Clinical Questionnaires and Scoring
  13. Integration with Dialogue and Reward.
  14. TD3 for Multi-Metric Interview Control, and Comparison to PPO and CEM
  15. Setting.
  16. ...and 19 more sections

Figures (7)

  • Figure 1: Repeated real-world sessions cause actuators and mechanical wear to a humanoid robot.
  • Figure 2: In-sim screenshot (UE5 MetaHuman patient runner). The pipeline uses AI-powered MetaHumans, enabling realistic facial animation for conversational diagnostics.
  • Figure 3: Closed loop for humanoid training with MetaHuman patients.
  • Figure 4: Learning stack shared by PPO, CEM, and our TD3 variant. Encoders feed a transformer fusion block; heads and updates differ by algorithm.
  • Figure 5: Learning curves for episodic reward (mean across runs). All methods improve across training and converge within a narrow band.
  • ...and 2 more figures