Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator

TL;DR

The proposed Koopman operator approach provides, for any practical purposes, equivalent results to the FNN-GH approach, thus bypassing the need to train a non-linear map and to use GH to extrapolate predictions in the ambient space and to solve the pre-image problem.

Abstract

We propose a machine-learning approach to model long-term out-of-sample dynamics of brain activity from task-dependent fMRI data. Our approach is a three stage one. First, we exploit Diffusion maps (DMs) to discover a set of variables that parametrize the low-dimensional manifold on which the emergent high-dimensional fMRI time series evolve. Then, we construct reduced-order-models (ROMs) on the embedded manifold via two techniques: Feedforward Neural Networks (FNNs) and the Koopman operator. Finally, for predicting the out-of-sample long-term dynamics of brain activity in the ambient fMRI space, we solve the pre-image problem coupling DMs with Geometric Harmonics (GH) when using FNNs and the Koopman modes per se. For our illustrations, we have assessed the performance of the two proposed schemes using a benchmark fMRI dataset with recordings during a visuo-motor task. The results suggest that just a few (for the particular task, five) non-linear coordinates of the high-dimensional fMRI time series provide a good basis for modelling and out-of-sample prediction of the brain activity. Furthermore, we show that the proposed approaches outperform the one-step ahead predictions of the naive random walk model, which, in contrast to our scheme, relies on the knowledge of the signals in the previous time step. Importantly, we show that the proposed Koopman operator approach provides, for any practical purposes, equivalent results to the FNN-GH approach, thus bypassing the need to train a non-linear map and to use GH to extrapolate predictions in the ambient fMRI space; one can use instead the low-frequency truncation of the DMs function space of L^2-integrable functions, to predict the entire list of coordinate functions in the fMRI space and to solve the pre-image problem.

Figures (5)

• Figure 1: Schematic of the proposed machine learning based methodology for modelling and predicting the dynamics of brain activity from task-dpended fMRI data.
• Figure 2: Thresholded image of the contrast "attention" + "non-attention" + "stationary" vs "fixation" is presented as an overlay on the reference structural MRI (T1) image.A) Level of significance p<0.05 (FWE corrected), B) $p<0.001$ (uncorrected).
• Figure 3: The eigenspectrum of DMs on the 111 time series that correspond to different brain regions. The red vertical line indicates how many eigenvectors we computed. The five parsimonious eigenvectors that were finally used as macroscopic variables are also marked with red arrows.
• Figure 4: Predictions based on the 5 parsimonious eigenvectors (namely $\mathbf{\psi}_1, \mathbf{\psi}_5, \mathbf{\psi}_8, \mathbf{\psi}_{13}$ and $\mathbf{\psi}_{15}$) when applying FNNs and the Koopman operator. Actual points for each one of the DMs are presented with black color up to the 280th point, which is the last point of the training set. The predicted values are presented with light green color up to the end of the time series (as derived iteratively by the ROMs from the point 281 to the end.).
• Figure 5: Predictions in the amplitude of the BOLD signal in the original fMRI space for four of the most "activated" regions as found by the classical GLM methodology A) left Calcarine Sulcus, B) left Middle Occipital Gyrus, C) right Lingual Gyrus, D) left Superior Occipital Gyrus. The red color marks the actual values of the test set, while the other colors correspond to the predictions based on the proposed methodology.