Few-Shot Transfer Learning for Individualized Braking Intent Detection on Neuromorphic Hardware

TL;DR

This work presents an energy-efficient, few-shot transfer learning method that is implemented on a neuromorphic processor capable of training a CSNN as new data becomes available, operating conditions change, or to customize group-level models to yield personalized models unique to each individual.

Abstract

Objective: This work explores use of a few-shot transfer learning method to train and implement a convolutional spiking neural network (CSNN) on a BrainChip Akida AKD1000 neuromorphic system-on-chip for developing individual-level, instead of traditionally used group-level, models using electroencephalographic data. Main Results: Efficacy of the above methodology to develop individual-specific braking intention predictive models by rapidly adapting the group-level model in as few as three training epochs while achieving at least 90% accuracy, true positive rate and true negative rate is presented. Further, results show the energy-efficiency of the neuromorphic hardware through a power reduction of over 97% with only a $1.3* increase in latency when using the Akida AKD1000 processor for network inference compared to an Intel Xeon central processing unit. Similar results were obtained in a subsequent ablation study using a subset of five out of 19 channels.

Figures (11)

• Figure 1: Schematic of the 10-20 configuration of EEG electrodes used for collecting data in this study. Grouping of the EEG electrodes correspond to frontal, parietal, temporal and occipital brain lobes Rojas_Alvarez_Montoya_de_la_Iglesia-Vaya_Cisternas_Gálvez_2018.
• Figure 2: Photograph of the experimental setup featuring several jetbots on a portion of the testbed, a participant wearing the Enobio 20 EEG headset, the steering wheel to navigate a jetbot, and the computer monitor on which the camera feed is displayed. Insert in the photograph shows Experiment 3 set-up. Note that although multiple jetbots are shown, only the jetbot controlled by the participant is present on the testbed during the experiments in this study. Photograph credit to Micheal Pierce/Missouri S&T.
• Figure 4: Experiment 3 data with markers for yellow and red traffic lights. Black line is the Cz grand average of all individuals across all trials. Blue lines are Cz averages for each of the 11 individual participants across all trials. Only data collected during the time period that is shaded in red or green is used for analysis. (a) Case where participants stopped at the red light. (b) Case where participants ignored and drove through the red light without stopping.
• Figure 5: Schematic of the few-shot transfer learning method, which incorporates a three-step approach; the first two sub-steps creates a group-level model on the CPU hardware (off-chip learning), and the third step maps this model to the Akida NsoC and implements few-shot learning on the individual-level data to create individual-level models (on-chip learning).
• Figure 7: Experiment 1 preprocessed EEG signals. (a) Channel potentials with associated countdown and “Stop” command markers and scale. Solid lines are the grand averages of all individuals across all trials. The shaded regions represent the range from minimum to maximum values across all trials (drawn at 1/40 scale of the grand average). (b) Black line is the Cz grand average signal of all individuals across all trials with scalp maps representing the grand average at the midpoint between two neighboring markers. Blue lines are Cz averages for each of the 11 individual participants across all trials. Color bar on the right displays the Cz channel potential in µV (adapted from "Convolutional spiking neural networks for intent detection based on anticipatory brain potentials using electroencephalogram" by Nathan Lutes, Venkata Sriram Siddhardh Nadendla & K. Krishnamurthy Lutes_Nadendla_Krishnamurthy_2024), used under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/) - scalp maps redrawn using different scale).