Towards Synthetic Data Generation for Improved Pain Recognition in Videos under Patient Constraints

TL;DR

This paper tackles the scarcity and ethical concerns of collecting real-world pain expressions in video by introducing a synthetic data generation pipeline. It extracts 3D facial meshes from a small participant set, maps diverse textures onto public-domain faces, and renders multi-view synthetic head videos to create a privacy-preserving dataset with 8,600 per perspective. When combined with real data, the synthetic dataset improves pain-recognition performance (e.g., AUROC up to 0.780 and F1 up to 0.817), though synthetic data alone is less effective than real data. The work demonstrates that synthetic data can augment real datasets to enhance generalization and inclusivity, and it provides an open-source resource to spur further privacy-preserving research in pain recognition.

Abstract

Recognizing pain in video is crucial for improving patient-computer interaction systems, yet traditional data collection in this domain raises significant ethical and logistical challenges. This study introduces a novel approach that leverages synthetic data to enhance video-based pain recognition models, providing an ethical and scalable alternative. We present a pipeline that synthesizes realistic 3D facial models by capturing nuanced facial movements from a small participant pool, and mapping these onto diverse synthetic avatars. This process generates 8,600 synthetic faces, accurately reflecting genuine pain expressions from varied angles and perspectives. Utilizing advanced facial capture techniques, and leveraging public datasets like CelebV-HQ and FFHQ-UV for demographic diversity, our new synthetic dataset significantly enhances model training while ensuring privacy by anonymizing identities through facial replacements. Experimental results demonstrate that models trained on combinations of synthetic data paired with a small amount of real participants achieve superior performance in pain recognition, effectively bridging the gap between synthetic simulations and real-world applications. Our approach addresses data scarcity and ethical concerns, offering a new solution for pain detection and opening new avenues for research in privacy-preserving dataset generation. All resources are publicly available to encourage further innovation in this field.

Figures (4)

• Figure 1: We leverage facial features extracted from a small dataset with real recordings with people in pain for generating a synthetic dataset with diverse faces and viewpoints to address logistical, ethical and bias-related challenges of such data collection.
• Figure 2: Generation Pipeline of Synthetic Head Videos. We extract facial textures from public domain faces for later translation. In parallel, we extract 3D volumes for each frame of the pain videos. Finally, we map the facial textures onto each frame-wise volume and render a scene from different perspectives.
• Figure 3: Example of different textures that are translated onto a participant of the BioVidHeat dataset.
• Figure 4: We consider three training configurations in our experiments: (1) using real data only (real to real), (2) using synthetic data exclusively (synth to real) and using both for training (mixed to real).