Sequence models for by-trial decoding of cognitive strategies from neural data

TL;DR

The paper addresses trial-to-trial variability in cognitive strategies during decision-making by decoding sequences of cognitive operations from EEG at the single-trial level. It combines Hidden Multivariate Pattern (HMP) analysis with a Structured State Space Sequence (S4) model to predict onset probabilities of cognitive operations across time within a trial, revealing an additional 'Confirmation' operation that is more prevalent under accuracy instructions. The approach links the occurrence of Confirmation to higher probability of correct responses and EMG markers of changes of mind, and replicates in a second SAT dataset, demonstrating robust trial-level decoding of cognitive strategies. This work highlights dynamic within-condition heterogeneity in decision strategies and showcases sequence modeling as a powerful tool for uncovering trial-level cognitive processes with broad potential applications.

Abstract

Understanding the sequence of cognitive operations that underlie decision-making is a fundamental challenge in cognitive neuroscience. Traditional approaches often rely on group-level statistics, which obscure trial-by-trial variations in cognitive strategies. In this study, we introduce a novel machine learning method that combines Hidden Multivariate Pattern analysis with a Structured State Space Sequence model to decode cognitive strategies from electroencephalography data at the trial level. We apply this method to a decision-making task, where participants were instructed to prioritize either speed or accuracy in their responses. Our results reveal an additional cognitive operation, labeled Confirmation, which seems to occur predominantly in the accuracy condition but also frequently in the speed condition. The modeled probability that this operation occurs is associated with higher probability of responding correctly as well as changes of mind, as indexed by electromyography data. By successfully modeling cognitive operations at the trial level, we provide empirical evidence for dynamic variability in decision strategies, challenging the assumption of homogeneous cognitive processes within experimental conditions. Our approach shows the potential of sequence modeling in cognitive neuroscience to capture trial-level variability that is obscured by aggregate analyses. The introduced method offers a new way to detect and understand cognitive strategies in a data-driven manner, with implications for both theoretical research and practical applications in many fields.

Figures (6)

• Figure 1: Conceptual representation of the introduced method. a): EEG data is fed into an HMP model b), which estimates cognitive operations per condition and outputs per-trial probabilities of the activation peak of each cognitive operation. HMP probabilities b) and raw EEG data a) are then processed using an S4 model c), where $A$, $B$, $C$, and $D$ are parameter matrices with different effects on the current state $h$, based on their position in the model. d): The S4 model outputs probabilities with loosened restrictions compared to HMP, providing the probability of the onset of a cognitive operation at each time step. See text for details.
• Figure 2: The result of the HMP fit reveals an additional event in the accuracy condition that is not present in the speed condition. Operation topographies are aligned to the (on average) most likely point in time at which each activation peak occurs. The brain activity at this point is averaged over all trials and participants to create the content of the topographies.
• Figure 3: a) The model (solid lines) predicts HMP probabilities (dashed line) well at single trial level. Shown here is one representative trial for each condition. b) An aggregate measure of true peak timing (Y-axis) and predicted peak timing (X-axis), values closer to the diagonal indicate a better prediction. c) The probability of correct response (Y-axis) increases with ACP (X-axis). Probability of second EMG event also increases with ACP. Data split into tertiles based on ACP for visualization purposes.
• Figure 4: HMP re-fit based on ACP tertiles, in accuracy trials with low ACP, the Confirmation operation disappears, while in speed trials with high ACP, the Confirmation operation appears. These findings oppose the initial fit, indicating that trials in these subsets were more likely to have followed the opposite strategy.
• Figure 5: The model architecture used, blue indicates data, green indicates processing, yellow indicates output. First, spatial features are extracted from raw data, followed by temporal dropout. Then, temporal convolution is used to model temporal relationships at multiple time scales, after which positional encoding is added to the features. Finally, all features are fed into a Mamba sequence model and classifier.