Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detection of Intoxicated Individuals from Facial Video Sequences via a Recurrent Fusion Model

Bita Baroutian, Atefe Aghaei, Mohsen Ebrahimi Moghaddam

TL;DR

The paper tackles non-invasive intoxication detection from video by proposing a recurrent fusion model that combines GAT-based facial landmark dynamics with a 3D-ResNet spatiotemporal feature stream. It introduces a shot-based processing workflow and a learnable adaptive fusion to robustly integrate modalities. A new YouTube-derived dataset of 3,542 clips from 202 individuals supports training and evaluation, with strong results showing 95.82% accuracy and 0.977 precision/recall, outperforming 3D-CNN and VGGFace+LSTM baselines. The approach demonstrates practical potential for public-safety monitoring while highlighting considerations around demographics, interpretability, and future extensions to related affective or safety-critical tasks.

Abstract

Alcohol consumption is a significant public health concern and a major cause of accidents and fatalities worldwide. This study introduces a novel video-based facial sequence analysis approach dedicated to the detection of alcohol intoxication. The method integrates facial landmark analysis via a Graph Attention Network (GAT) with spatiotemporal visual features extracted using a 3D ResNet. These features are dynamically fused with adaptive prioritization to enhance classification performance. Additionally, we introduce a curated dataset comprising 3,542 video segments derived from 202 individuals to support training and evaluation. Our model is compared against two baselines: a custom 3D-CNN and a VGGFace+LSTM architecture. Experimental results show that our approach achieves 95.82% accuracy, 0.977 precision, and 0.97 recall, outperforming prior methods. The findings demonstrate the model's potential for practical deployment in public safety systems for non-invasive, reliable alcohol intoxication detection.

Detection of Intoxicated Individuals from Facial Video Sequences via a Recurrent Fusion Model

TL;DR

The paper tackles non-invasive intoxication detection from video by proposing a recurrent fusion model that combines GAT-based facial landmark dynamics with a 3D-ResNet spatiotemporal feature stream. It introduces a shot-based processing workflow and a learnable adaptive fusion to robustly integrate modalities. A new YouTube-derived dataset of 3,542 clips from 202 individuals supports training and evaluation, with strong results showing 95.82% accuracy and 0.977 precision/recall, outperforming 3D-CNN and VGGFace+LSTM baselines. The approach demonstrates practical potential for public-safety monitoring while highlighting considerations around demographics, interpretability, and future extensions to related affective or safety-critical tasks.

Abstract

Alcohol consumption is a significant public health concern and a major cause of accidents and fatalities worldwide. This study introduces a novel video-based facial sequence analysis approach dedicated to the detection of alcohol intoxication. The method integrates facial landmark analysis via a Graph Attention Network (GAT) with spatiotemporal visual features extracted using a 3D ResNet. These features are dynamically fused with adaptive prioritization to enhance classification performance. Additionally, we introduce a curated dataset comprising 3,542 video segments derived from 202 individuals to support training and evaluation. Our model is compared against two baselines: a custom 3D-CNN and a VGGFace+LSTM architecture. Experimental results show that our approach achieves 95.82% accuracy, 0.977 precision, and 0.97 recall, outperforming prior methods. The findings demonstrate the model's potential for practical deployment in public safety systems for non-invasive, reliable alcohol intoxication detection.

Paper Structure

This paper contains 25 sections, 4 equations, 9 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Physiological Measurement Techniques
  4. Machine Learning and Vision-Based Techniques
  5. Methodology
  6. Overview of the framework
  7. Shot Change Detection and Preprocessing
  8. Facial Landmark Extraction and Graph-Based Analysis
  9. Spatiotemporal Visual Feature Extraction
  10. Adaptive Feature Fusion
  11. Experimental Results
  12. Dataset
  13. Implementation Details
  14. Evaluation
  15. Ablation Studies
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Architecture of the proposed model.
  • Figure 2: Architecture of the TransNetV2 model.
  • Figure 3: Architecture of SPIGA.
  • Figure 4: The 68 facial landmark points and their positioning
  • Figure 5: Sample frames from our intoxication detection dataset
  • ...and 4 more figures