Block-As-Domain Adaptation for Workload Prediction from fNIRS Data

TL;DR

The paper tackles the challenge of predicting cognitive workload from fNIRS data in settings with unseen subjects and sessions. It introduces Class-Aware Block-Aware Domain Adaptation (CABA-DA), combining a block-level discrepancy term and a contrastive domain discrepancy to align intra-class distributions across blocks and sessions, alongside a modified MLPMixer backbone tailored for fNIRS signals. The overall objective, $l = l^{ce} + \alpha(D^{cdd} + D^{caba})$, balances supervised learning with domain alignment, with $\alpha$ tuned by grid search. Across TUberlin, Tufts, and FFT datasets, CABA-DA yields consistent improvements over baseline models and demonstrates that block-wise domain shifts are a crucial source of variance; the approach remains computationally efficient at inference and provides practical benefits for real-world CWL monitoring and HCI systems.

Abstract

Functional near-infrared spectroscopy (fNIRS) is a non-intrusive way to measure cortical hemodynamic activity. Predicting cognitive workload from fNIRS data has taken on a diffuse set of methods. To be applicable in real-world settings, models are needed, which can perform well across different sessions as well as different subjects. However, most existing works assume that training and testing data come from the same subjects and/or cannot generalize well across never-before-seen subjects. Additional challenges imposed by fNIRS data include the high variations in inter-subject fNIRS data and also in intra-subject data collected across different blocks of sessions. To address these issues, we propose an effective method, referred to as the class-aware-block-aware domain adaptation (CABA-DA) which explicitly minimize intra-session variance by viewing different blocks from the same subject same session as different domains. We minimize the intra-class domain discrepancy and maximize the inter-class domain discrepancy accordingly. In addition, we propose an MLPMixer-based model for cognitive load classification. Experimental results demonstrate the proposed model has better performance compared with three different baseline models on three public-available datasets of cognitive workload. Two of them are collected from n-back tasks and one of them is from finger tapping. From our experiments, we also show the proposed contrastive learning method can also improve baseline models we compared with.

Figures (6)

• Figure 1: Commonly used experiment design when collecting fNIRS data. Different colors (best view in color) show different sessions at the top part, and the orange color indicating trails.
• Figure 2: Four scenarios of splits.
• Figure 3: The MLPMixer-based classifier that we adapted for the fNIRS data. The input data $\hat{x} \in \mathbb{R}^{2 \times T \times D}$ is first encoded by an FC layer ($\mathbb{R}^{2\times D} \mapsto \mathbb{R}^{C}$). The remainder of this figure is the same as the original MLPMixer tolstikhin2021mlp: the encoded feature ($F$) is sent to $N$ mixer layers. Each mixer layer contains one temporal-mixing MLP ( $\mathbb{R}^T \mapsto \mathbb{R}^T$) and one channel-mixing MLP ( $\mathbb{R}^C \mapsto \mathbb{R}^C$). An MLP has two FC layers and a GELU nonlinearity. Other components include layer normalization, skip-connection and global average pooling.
• Figure 4: The fNIRS channels and masks for the (a) TUBerlin and (b) FFT dataset. The red dots represent the fNIRS channels and the masks are numbered. There are 36 channels and 10 masks for the TUBerlin dataset, and 20 channels and 6 masks for the FFT dataset. Each mask covers 4 channels.
• Figure 5: Plot of of masking one position at a time using (a) MLPBiGRU-Block-DA on TUBerlin and (b) MLPMixer-Block-DA on FFT. The x-axis shows the position of the mask and the y-axis shows the k-fold cross validation accuracy on test splits. The black dotted line represents the average performance across all mask positions. The red dash line shows the performance without any mask. The green and blue dot-dash lines indicate the lower and upper bounds of the 95% confidence intervals of the masked accuracy.