Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Session Decoding of Neural Spiking Data via Task-Conditioned Latent Alignment

Canyang Zhao, Bolin Peng, J. Patrick Mayo, Ce Ju, Bing Liu

TL;DR

This work tackles cross-session nonstationarity in invasive BCI neural signals by introducing Task-Conditioned Latent Alignment (TCLA). TCLA learns source-session latent neural dynamics with a shared autoencoder and aligns target sessions per task condition using multi-kernel MMD, enabling robust decoding with limited data. Evaluated on three non-human primate center-out datasets spanning arm and eye movements, TCLA yields consistent decoding improvements (e.g., up to $R^2$ gains of $0.386$) and produces aligned latent manifolds as visualized by t-SNE, indicating successful cross-session transfer. The method provides a practical pathway to more stable, long-term BCI operation by leveraging previously collected data to calibrate new sessions without extensive retraining.

Abstract

Cross-session nonstationarity in neural activity recorded by implanted electrodes is a major challenge for invasive Brain-computer interfaces (BCIs), as decoders trained on data from one session often fail to generalize to subsequent sessions. This issue is further exacerbated in practice, as retraining or adapting decoders becomes particularly challenging when only limited data are available from a new session. To address this challenge, we propose a Task-Conditioned Latent Alignment framework (TCLA) for cross-session neural decoding. Building upon an autoencoder architecture, TCLA first learns a low-dimensional representation of neural dynamics from a source session with sufficient data. For target sessions with limited data, TCLA then aligns target latent representations to the source in a task-conditioned manner, enabling effective transfer of learned neural dynamics. We evaluate TCLA on the macaque motor and oculomotor center-out dataset. Compared to baseline methods trained solely on target-session data, TCLA consistently improves decoding performance across datasets and decoding settings, with gains in the coefficient of determination of up to 0.386 for y coordinate velocity decoding in a motor dataset. These results suggest that TCLA provides an effective strategy for transferring knowledge from source to target sessions, enabling more robust neural decoding under conditions with limited data.

Cross-Session Decoding of Neural Spiking Data via Task-Conditioned Latent Alignment

TL;DR

This work tackles cross-session nonstationarity in invasive BCI neural signals by introducing Task-Conditioned Latent Alignment (TCLA). TCLA learns source-session latent neural dynamics with a shared autoencoder and aligns target sessions per task condition using multi-kernel MMD, enabling robust decoding with limited data. Evaluated on three non-human primate center-out datasets spanning arm and eye movements, TCLA yields consistent decoding improvements (e.g., up to gains of ) and produces aligned latent manifolds as visualized by t-SNE, indicating successful cross-session transfer. The method provides a practical pathway to more stable, long-term BCI operation by leveraging previously collected data to calibrate new sessions without extensive retraining.

Abstract

Cross-session nonstationarity in neural activity recorded by implanted electrodes is a major challenge for invasive Brain-computer interfaces (BCIs), as decoders trained on data from one session often fail to generalize to subsequent sessions. This issue is further exacerbated in practice, as retraining or adapting decoders becomes particularly challenging when only limited data are available from a new session. To address this challenge, we propose a Task-Conditioned Latent Alignment framework (TCLA) for cross-session neural decoding. Building upon an autoencoder architecture, TCLA first learns a low-dimensional representation of neural dynamics from a source session with sufficient data. For target sessions with limited data, TCLA then aligns target latent representations to the source in a task-conditioned manner, enabling effective transfer of learned neural dynamics. We evaluate TCLA on the macaque motor and oculomotor center-out dataset. Compared to baseline methods trained solely on target-session data, TCLA consistently improves decoding performance across datasets and decoding settings, with gains in the coefficient of determination of up to 0.386 for y coordinate velocity decoding in a motor dataset. These results suggest that TCLA provides an effective strategy for transferring knowledge from source to target sessions, enabling more robust neural decoding under conditions with limited data.
Paper Structure (14 sections, 6 equations, 4 figures)

This paper contains 14 sections, 6 equations, 4 figures.

Table of Contents

  1. INTRODUCTION
  2. METHODS
  3. Model Architecture
  4. Two-Stage Training Framework
  5. Source Session Pre-training
  6. Target Session Alignment
  7. EXPERIMENTS
  8. Datasets
  9. Experimental Settings
  10. RESULTS
  11. Conditional Aligned Latent Manifolds
  12. Decoding Performance
  13. DISCUSSION
  14. CONCLUSION

Figures (4)

  • Figure 1: Architecture of the proposed model. The model is composed of a shared autoencoder module and session-specific layers. Neural spiking data from a source session with sufficient trials is used to learn a latent representation via the encoder module. For target sessions with limited trials, alignment is performed separately for each task condition (movement direction in the center-out task) using multi-kernel MMD in latent space.
  • Figure 2: Experimental Paradigm. (a) Eight-direction motor center-out task. Two monkeys perform reaching tasks while neural activity is recorded from M1. (b) Four-direction oculomotor center-out task. A monkey performs gaze-tracking tasks while neural activity is recorded from FEF and MT.
  • Figure 3: T-SNE visualization of the latent representations from the source and target sessions. Each column corresponds to one dataset ($MOTORCO_1$, $MOTORCO_2$, or $OCULOCO_1$). The top row shows the latent representations learned by LDNSws, where the autoencoder is trained independently for each session without cross-session latent alignment. The bottom row shows the latent representations learned by TCLA. The manifolds of target sessions are merged into a cohesive cluster aligned with the manifold of the source session.
  • Figure 4: Comparison of decoding performance across methods and Datasets. Each column represents a different dataset. The top row shows $R^2$ for position decoding, and the bottom row shows the results for velocity decoding. For each method and kinematic prediction, performance is shown separately for the $x$ (blue) and $y$ (green) coordinates. Each dot represents the mean $R^2$ of a target session. The solid black square marker indicates the bootstrap mean, and the error bar represents the 95% CI across all target sessions.