ViF-SD2E: A Robust Weakly-Supervised Method for Neural Decoding

TL;DR

This paper addresses neural decoding under noisy or weak labels by introducing ViF-SD2E, a robust framework that combines a space-division (SD) module with an exploration--exploitation (2E) strategy guided by weak 0/1 vision-feedback (ViF) labels. The method uses an unsupervised EM-based exploration to generate predictions, then corrects them via SD before exploiting them with a time-series model (LSTM), enabling open-loop or closed-loop training and a controllable supervision parameter $N$. Empirical results on macaque finger-movement data show that ViF-SD2E can reach or closely approach supervised performance, with strong robustness and notable improvements when $N$ is tuned (e.g., $N=3$). The work contributes a universal ViF and SD framework, a novel interaction learning paradigm with 2E, and demonstrates effective weak-label neural decoding with potential broader impacts on regression under weak supervision.

Abstract

Neural decoding plays a vital role in the interaction between the brain and the outside world. In this paper, we directly decode the movement track of a finger based on the neural signals of a macaque. Supervised regression methods may overfit to actual labels containing noise, and require a high labeling cost, while unsupervised approaches often have unsatisfactory accuracy. Besides, the spatial and temporal information is often ignored or not well exploited by those methods. This motivates us to propose a robust weakly-supervised method, called ViF-SD2E, for neural decoding. In particular, it consists of a space-division (SD) module and a exploration--exploitation (2E) strategy, to effectively exploit both the spatial information of the outside world and the temporal information of neural activity, where the SD2E output is analogized with the weak 0/1 vision-feedback (ViF) label for training. It is worth noting that the designed ViF-SD2E is based on a symmetric phenomenon between the unsupervised decoding trajectory and the real trajectory in previous observations, then a cognitive pattern of fuzzy (robust) interaction in the nervous system may be discovered by us. Extensive experiments demonstrate the effectiveness of our method, which can be sometimes comparable to supervised counterparts.

Figures (10)

• Figure 1: Open-loop (a) and closed-loop (b) optimization procedures of the proposed ViF-SD2E method. Here, the red arrow and the cyan arrow represent the input of vision-feedback from the outside world and the input of neural signals from the brain, respectively. The solid arrow represents the data transmission process and the dashed arrow represents interactive and evaluative feedback, which constitute the training or learning process of the ViF-SD2E. $\textbf{S}_k$ is the neural signal after time series processing. $z_k$ is the true label. $\hat{z}_k$ is the value decoded by unsupervised exploration. $\bar{z}_k$ is the value corrected by SD module. $\tilde{z}_k$ is the value output by supervised exploitation and is used to update the weights of exploration. $k$ is the $k$th moment or the $k$th sample. The bits (0/1) are the converted symbols after being decoded by SD2E (neural signals $\rightarrow$ value decoded $\rightarrow$ 0/1), and then analogized with the symbols (0 or 1) provided from ViF.
• Figure 2: The movement space of the finger that interacts with the brain. Assume that the blue ($N=1$) and orange ($N=2$) area are the active space in a certain decoding. The $A\bullet$ is the real label, and $A'_0\bullet$, $A'_1\bullet$, $A'_2\bullet$ or $A'_3\bullet$ are the unsupervised predicted values that may appear after the exploration. Step 1 ($N=1$): The 2-D space is divided equally, and their intersection is $O'$. Then four subspaces, $\mathbb R^{(1)}_{0,0}(0, 0)$, $\mathbb R^{(1)}_{0,1}(0, 1)$, $\mathbb R^{(1)}_{0,2}(1, 0)$ and $\mathbb R^{(1)}_{0,3}(1, 1)$, are demarcated. Next, we use spatial symmetry to keep the predicted values and the true labels in one subspace, and obtain $\bullet A"_0$, $\bullet A"'_1$, $\bullet A"_2$ and $\bullet A'_3$. Step 2 ($N=2$): Repeat step 1: $\mathbb R^{(1)}_{0,2}(1, 0)$ is divided into four subspaces, $\mathbb R^{(2)}_{2,0}(0, 0)$, $\mathbb R^{(2)}_{2,1}(0, 1)$, $\mathbb R^{(2)}_{2,2}(1, 0)$, $\mathbb R^{(2)}_{2,3}(1, 1)$. Here, $A\bullet$ is located in $\mathbb R^{(2)}_{2,2}(1, 0)$. Then we also employ spatial symmetry to keep them in one subspace, and get $\bullet A"_0$, $\bullet A""_1$, $\bullet A"_2$ and $\bullet A'_3$. As $N\rightarrow + \infty$, our predicted values will gradually approach the true labels.
• Figure 3: Intuitive diagram of space division in the movement space.
• Figure 4: The framework of the proposed ViF-SD2E. In the training phase: 1) the neural signals that have a one-to-one correspondence with the movement of the finger are collected, and the movements are encoded as 0/1 signals via the ViF; the neural signals corresponding to each sample are preprocessed as feature $\textbf{S}_{k}$ together with a time sequence, which is the input to our ViF-SD2E. 2) Then, the $\overline z_{1:K}$ predicted by the exploration are fed into the designed SD module, where they are encoded as $\overline z_{1:K, bit}$ and compared with the given $z_{1:K, bit}$. After being processed by global or local methods, the $\overline z_{1:K}$ is corrected to $\tilde{z}_{1:K}$; 3) in the exploitation; the $\tilde{z}_{1:K}$ are then used as ground truth to train the exploitation together with the input feature ${\bf S}_{1:K}$. Finally, the trained exploitation is adopted for testing and further used to evaluate the $N$-confidence.
• Figure 5: The red box represents the smallest processing unit in the global and local methods. (a) The steps of the global method. What the global method is concerned about is that the unsupervised predicted $\overline x^{(0)}_{1:K}$ and $\overline y^{(0)}_{1:K}$ are sent directly to the equation (1)-(2) for coding and correction. Furthermore, the output of the global method is the corrected $\tilde{x}^{N}_{1:K}$ and $\tilde{y}^{N}_{1:K}$. (b) The processing unit of the global method. $\overline x^{(1)}_{1:k_1, left}$ indicates that in the first division, the samples of the $x$-axis in the left subspace are taken, and the total number of samples is $k_1$. Next, $\overline x^{(1)}_{1:k_1, left}$ is coded as 0/1 by $F^{(1)}_{bit}$ and compared with the given 0/1, and then corrected by $F^{(1)}_{update}$. Finally, the output is $\overline x^{(2)}_{1:k_1, left}$. (c) The steps of the local method. (d) The processing unit of the local method. The biggest difference between local and global is that in the local unit, the neural signal $\textbf{S}^{(1)}_{1:k_1, left}$ corresponding to $\overline x^{(1)}_{1:k_1, left}$ is processed by an unsupervised algorithm (Un-EM) to obtain $\overline x'^{(1)}_{1:k_1, left}$, instead of the source $\overline x^{(1)}_{1:k_1, left}$.