Interpersonal Relationship Analysis with Dyadic EEG Signals via Learning Spatial-Temporal Patterns

TL;DR

The paper addresses objective quantification of interpersonal relationships from dyadic EEG data collected during co-viewing of positive-emotion videos. It introduces DSEN, a two-branch architecture that combines temporal CNN features with a spatial Dynamic Graph Convolutional Network on 30 EEG channels, and trains with a triplet loss and a CCA loss to produce discriminative, correlated dyadic representations. A new dyadic-subject EEG dataset (72 pairs; strangers vs friends) is constructed, with nine video clips and 2-second windows, enabling robust evaluation. Experimental results show DSEN outperforms baselines by substantial margins and ablations confirm the importance of each component, highlighting the framework’s potential for objective assessment of team dynamics and therapeutic interactions.

Abstract

Interpersonal relationship quality is pivotal in social and occupational contexts. Existing analysis of interpersonal relationships mostly rely on subjective self-reports, whereas objective quantification remains challenging. In this paper, we propose a novel social relationship analysis framework using spatio-temporal patterns derived from dyadic EEG signals, which can be applied to quantitatively measure team cooperation in corporate team building, and evaluate interpersonal dynamics between therapists and patients in psychiatric therapy. First, we constructed a dyadic-EEG dataset from 72 pairs of participants with two relationships (stranger or friend) when watching emotional videos simultaneously. Then we proposed a deep neural network on dyadic-subject EEG signals, in which we combine the dynamic graph convolutional neural network for characterizing the interpersonal relationships among the EEG channels and 1-dimension convolution for extracting the information from the time sequence. To obtain the feature vectors from two EEG recordings that well represent the relationship of two subjects, we integrate deep canonical correlation analysis and triplet loss for training the network. Experimental results show that the social relationship type (stranger or friend) between two individuals can be effectively identified through their EEG data.

Figures (9)

• Figure 1: The procedure of exploring the relationships between two individuals. In our proposed framework, the relationship between the two participants (stranger or friend) can be determined based on their EEG recordings while watching emotional video clips.
• Figure 2: The framework of our proposed Dyadic-Subject EEG Network (DSEN). DSEN takes dyadic-subject EEG data as input to extract temporal and spatial features with the feature extractor $f$. The features are fused by an attention-based module to form a cohesive representation for classification. The feature extractor $f$ is trained individually via a triplet loss. The fused features are utilized by a classifier, which is trained using a combined CCA loss and classification loss.
• Figure 3: The trial procedure of our dyadic-subject EEG data collection.
• Figure 4: (a) is the distribution plot of the unprocessed dyadic EEG data in a two-dimensional plane after dimension reduction by t-SNE. (b) is the distribution plot of extracted and fused features from the dyadic EEG data in the plane after dimension reduction by t-SNE.
• Figure 5: The detailed framework of the feature extractor in our proposed DSEN.