Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation

Mina Bishay, Graham Page, Mohammad Mavadati

TL;DR

PainNet tackles the under-explored problem of sequence-level self-reported pain estimation by learning a relational, end-to-end model that compares video pairs. It combines an embedding pipeline (AFFDEX-based AU predictions, temporal GRU, and a statistical layer) with a relation module that uses metric learning to predict pain-level similarity, trained through episode-based sampling. The approach achieves state-of-the-art performance on the UNBC dataset in ICC and MAE and provides balanced error across pain intensities, demonstrating robustness in data-limited settings. This work advances automatic clinical pain assessment by enabling reliable, sequence-level predictions with a streamlined, end-to-end training scheme.

Abstract

Despite the span in estimating pain from facial expressions, limited works have focused on estimating the sequence-level pain, which is reported by patients and used commonly in clinics. In this paper, we introduce a novel Statistical Relation Network, referred to as PainNet, designed for the estimation of the sequence-level pain. PainNet employs two key modules, the embedding and the relation modules, for comparing pairs of pain videos, and producing relation scores indicating if each pair belongs to the same pain category or not. At the core of the embedding module is a statistical layer mounted on the top of a RNN for extracting compact video-level features. The statistical layer is implemented as part of the deep architecture. Doing so, allows combining multiple training stages used in previous research, into a single end-to-end training stage. PainNet is trained using the episode-based training scheme, which involves comparing a query video with a set of videos representing the different pain categories. Experimental results show the benefit of using the statistical layer and the episode-based training in the proposed model. Furthermore, PainNet outperforms the state-of-the-art results on self-reported pain estimation.

PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation

TL;DR

PainNet tackles the under-explored problem of sequence-level self-reported pain estimation by learning a relational, end-to-end model that compares video pairs. It combines an embedding pipeline (AFFDEX-based AU predictions, temporal GRU, and a statistical layer) with a relation module that uses metric learning to predict pain-level similarity, trained through episode-based sampling. The approach achieves state-of-the-art performance on the UNBC dataset in ICC and MAE and provides balanced error across pain intensities, demonstrating robustness in data-limited settings. This work advances automatic clinical pain assessment by enabling reliable, sequence-level predictions with a streamlined, end-to-end training scheme.

Abstract

Despite the span in estimating pain from facial expressions, limited works have focused on estimating the sequence-level pain, which is reported by patients and used commonly in clinics. In this paper, we introduce a novel Statistical Relation Network, referred to as PainNet, designed for the estimation of the sequence-level pain. PainNet employs two key modules, the embedding and the relation modules, for comparing pairs of pain videos, and producing relation scores indicating if each pair belongs to the same pain category or not. At the core of the embedding module is a statistical layer mounted on the top of a RNN for extracting compact video-level features. The statistical layer is implemented as part of the deep architecture. Doing so, allows combining multiple training stages used in previous research, into a single end-to-end training stage. PainNet is trained using the episode-based training scheme, which involves comparing a query video with a set of videos representing the different pain categories. Experimental results show the benefit of using the statistical layer and the episode-based training in the proposed model. Furthermore, PainNet outperforms the state-of-the-art results on self-reported pain estimation.

Paper Structure

This paper contains 24 sections, 4 equations, 2 figures, 7 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Proposed Method
  4. Embedding Module
  5. Facial Expression Analysis
  6. Temporal modeling
  7. Statistical layer
  8. Relation Module
  9. Training scheme
  10. Episode-Based Training
  11. Loss function
  12. Experimental Results
  13. Dataset
  14. Performance metrics
  15. Implementation details
  16. ...and 9 more sections

Figures (2)

  • Figure 1: The proposed architecture for estimating the sequence-level pain score.
  • Figure 2: MAE per intensity for the PainNet and Trajectory Analysis szczapa2022automatic models.