Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Post-hoc and manifold explanations analysis of facial expression data based on deep learning

Yang Xiao

TL;DR

The paper tackles how deep networks process facial expression data to predict psychological attributes and how to explain these associations. It uses VGG16 to model facial representations, Grad-CAM for post-hoc visualization, and t-SNE/UMAP for manifold-based explanations, evaluated on the Bainbridge 10k US Adult Faces Database across 20 attributes. Key contributions include achieving strong predictive power, linking facial features to psychology via Grad-CAM, and revealing cluster structures (or their absence) in attribute manifolds, along with ethical considerations of bias. The work advances AI explainability in psychology and provides practical tools and code to study cognitive inferences from facial data.

Abstract

The complex information processing system of humans generates a lot of objective and subjective evaluations, making the exploration of human cognitive products of great cutting-edge theoretical value. In recent years, deep learning technologies, which are inspired by biological brain mechanisms, have made significant strides in the application of psychological or cognitive scientific research, particularly in the memorization and recognition of facial data. This paper investigates through experimental research how neural networks process and store facial expression data and associate these data with a range of psychological attributes produced by humans. Researchers utilized deep learning model VGG16, demonstrating that neural networks can learn and reproduce key features of facial data, thereby storing image memories. Moreover, the experimental results reveal the potential of deep learning models in understanding human emotions and cognitive processes and establish a manifold visualization interpretation of cognitive products or psychological attributes from a non-Euclidean space perspective, offering new insights into enhancing the explainability of AI. This study not only advances the application of AI technology in the field of psychology but also provides a new psychological theoretical understanding the information processing of the AI. The code is available in here: https://github.com/NKUShaw/Psychoinformatics.

Post-hoc and manifold explanations analysis of facial expression data based on deep learning

TL;DR

The paper tackles how deep networks process facial expression data to predict psychological attributes and how to explain these associations. It uses VGG16 to model facial representations, Grad-CAM for post-hoc visualization, and t-SNE/UMAP for manifold-based explanations, evaluated on the Bainbridge 10k US Adult Faces Database across 20 attributes. Key contributions include achieving strong predictive power, linking facial features to psychology via Grad-CAM, and revealing cluster structures (or their absence) in attribute manifolds, along with ethical considerations of bias. The work advances AI explainability in psychology and provides practical tools and code to study cognitive inferences from facial data.

Abstract

The complex information processing system of humans generates a lot of objective and subjective evaluations, making the exploration of human cognitive products of great cutting-edge theoretical value. In recent years, deep learning technologies, which are inspired by biological brain mechanisms, have made significant strides in the application of psychological or cognitive scientific research, particularly in the memorization and recognition of facial data. This paper investigates through experimental research how neural networks process and store facial expression data and associate these data with a range of psychological attributes produced by humans. Researchers utilized deep learning model VGG16, demonstrating that neural networks can learn and reproduce key features of facial data, thereby storing image memories. Moreover, the experimental results reveal the potential of deep learning models in understanding human emotions and cognitive processes and establish a manifold visualization interpretation of cognitive products or psychological attributes from a non-Euclidean space perspective, offering new insights into enhancing the explainability of AI. This study not only advances the application of AI technology in the field of psychology but also provides a new psychological theoretical understanding the information processing of the AI. The code is available in here: https://github.com/NKUShaw/Psychoinformatics.
Paper Structure (19 sections, 11 equations, 5 figures, 2 tables)

This paper contains 19 sections, 11 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Interdisciplinary background
  4. Post-hoc explanation
  5. Manifold Learning
  6. Preliminary
  7. Explanation
  8. Grad-CAM
  9. t-SNE
  10. UMAP
  11. Experiments
  12. Dataset
  13. Training
  14. Post-hoc explanation
  15. Manifold learning
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Heatmaps for human and neural network
  • Figure 2: Target: happy
  • Figure 3: t-SNE manifold visualization
  • Figure 4: UMAP manifold visualization
  • Figure 5: Target: attractive