Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks

Panfeng Li, Mohamed Abouelenien, Rada Mihalcea, Zhicheng Ding, Qikai Yang, Yiming Zhou

TL;DR

The results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data and propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance.

Abstract

Deception detection is gaining increasing interest due to ethical and security concerns. This paper explores the application of convolutional neural networks for the purpose of multimodal deception detection. We use a dataset built by interviewing 104 subjects about two topics, with one truthful and one falsified response from each subject about each topic. In particular, we make three main contributions. First, we extract linguistic and physiological features from this data to train and construct the neural network models. Second, we propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance. Third, we compare our new approach with earlier methods designed for multimodal deception detection. We find that our system outperforms regular classification methods; our results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data.

Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks

TL;DR

The results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data and propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance.

Abstract

Deception detection is gaining increasing interest due to ethical and security concerns. This paper explores the application of convolutional neural networks for the purpose of multimodal deception detection. We use a dataset built by interviewing 104 subjects about two topics, with one truthful and one falsified response from each subject about each topic. In particular, we make three main contributions. First, we extract linguistic and physiological features from this data to train and construct the neural network models. Second, we propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance. Third, we compare our new approach with earlier methods designed for multimodal deception detection. We find that our system outperforms regular classification methods; our results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data.
Paper Structure (16 sections, 8 figures)

This paper contains 16 sections, 8 figures.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Dataset
  4. Bimodal CNNs
  5. Vector Representations of Words
  6. Experimental setup
  7. Data Preprocessing
  8. Physiological Modality
  9. Linguistic Modality
  10. Training and Testing Procedures
  11. Experimental Results and Discussion
  12. Individual and integrated modalities
  13. Cross-Topic Learning
  14. Stability Analysis
  15. Compared with the Regular Models
  16. ...and 1 more sections

Figures (8)

  • Figure 1: BiModal CNN
  • Figure 2: Deception recall, truthfulness recall, and overall accuracy percentages for individual and integrated modalities using features extracted from the "Abortion" topic
  • Figure 3: Deception recall, truthfulness recall, and overall accuracy percentages for individual and integrated modalities using features extracted from the "Best Friend" topic
  • Figure 4: Deception recall, truthfulness recall, and overall accuracy percentages for individual and integrated modalities using features extracted from both the "Abortion" and "Best Friend" topic
  • Figure 5: Deception, truthfulness, and overall accuracy percentages for individual and integrated modalities using across-topic learning. "Abortion" features are used for training and "Best Friend" features are used for testing
  • ...and 3 more figures