Pain in 3D: Generating Controllable Synthetic Faces for Automated Pain Assessment

TL;DR

The paper tackles the lack of diverse, high-quality pain-expression data by introducing 3DPain, a diffusion-augmented synthetic pipeline that delivers 82,500 pain-annotated frames across 2,500 identities with precise AU and $PSPI$ annotations. It couples this with ViTPain, a cross-modal Vision Transformer that uses neutral-reference cross-attention to isolate dynamic pain signals from identity, enabling improved $PSPI$ regression and AU estimation. Across UNBC-McMaster benchmarks, synthetic pre-training with 3DPain yields significant generalization gains, underscoring the value of large-scale, clinically-grounded synthetic data for pain assessment. The work offers a scalable, interpretable framework for automated pain analysis while noting domain-shift in skin textures and the need for temporal modeling in future work.

Abstract

Automated pain assessment from facial expressions is crucial for non-communicative patients, such as those with dementia. Progress has been limited by two challenges: (i) existing datasets exhibit severe demographic and label imbalance due to ethical constraints, and (ii) current generative models cannot precisely control facial action units (AUs), facial structure, or clinically validated pain levels. We present 3DPain, a large-scale synthetic dataset specifically designed for automated pain assessment, featuring unprecedented annotation richness and demographic diversity. Our three-stage framework generates diverse 3D meshes, textures them with diffusion models, and applies AU-driven face rigging to synthesize multi-view faces with paired neutral and pain images, AU configurations, PSPI scores, and the first dataset-level annotations of pain-region heatmaps. The dataset comprises 82,500 samples across 25,000 pain expression heatmaps and 2,500 synthetic identities balanced by age, gender, and ethnicity. We further introduce ViTPain, a Vision Transformer based cross-modal distillation framework in which a heatmap-trained teacher guides a student trained on RGB images, enhancing accuracy, interpretability, and clinical reliability. Together, 3DPain and ViTPain establish a controllable, diverse, and clinically grounded foundation for generalizable automated pain assessment.

Figures (4)

• Figure 1: 3DPain synthetic dataset. Our dataset provides multiview images and corresponding heatmaps of facial expressions, enabling robust training and evaluation for automatic pain recognition.
• Figure 2: Pipeline for facial pain expression generation. A neutral FLAME identity is sampled and rendered as a frontal depth map, which conditions a depth-guided image generation model to synthesize a realistic face. The generated image is used to texture the neutral mesh, after which pain expressions are introduced via Action Unit–based rigging and texture transfer. The expressive mesh is rendered from multiple views and refined with segmentation-based background inpainting to produce the final images.
• Figure 3: Quantitative comparison of frames in the synthetic 3DPain dataset as well as the UNBC-McMaster dataset for PSPI scores between 0-16.
• Figure 4: Overview of the proposed ViTPain framework. A pain expression image is tokenized and encoded by a Vision Transformer to produce CLS and patch tokens, which cross-attend to patch tokens from a neutral reference image of the same identity. The resulting CLS token is used for PSPI regression and pain/no-pain classification. During training only, patch tokens additionally cross-attend to learnable AU query tokens to predict Action Unit intensities, providing auxiliary supervision.