Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats

TL;DR

The study develops a multi-modal seizure-detection framework for laboratory rats by separately modeling ECoG, Piezo, and video data and then fusing their predictions through postprocessing. It employs a Time Series Forest and an RNN for ECoG and Piezo, and VideoMAE for video clips, with undersampling and thresholding to address severe class imbalance, and timestamp-aligned ensemble fusion to output seizure onset and duration. Results show that while single modalities suffer from high false positives, the combined system achieves high recall and improved precision, with inference times compatible with real-time operation. The approach demonstrates that multi-modality can clean noisy signals and boost detection performance, offering a practical tool for automated monitoring and pharmacological assessments in preclinical epilepsy research.

Abstract

A multi-modal machine learning system uses multiple unique data sources and types to improve its performance. This article proposes a system that combines results from several types of models, all of which are trained on different data signals. As an example to illustrate the efficacy of the system, an experiment is described in which multiple types of data are collected from rats suffering from seizures. This data includes electrocorticography readings, piezoelectric motion sensor data, and video recordings. Separate models are trained on each type of data, with the goal of classifying each time frame as either containing a seizure or not. After each model has generated its classification predictions, these results are combined. While each data signal works adequately on its own for prediction purposes, the significant imbalance in class labels leads to increased numbers of false positives, which can be filtered and removed by utilizing all data sources. This paper will demonstrate that, after postprocessing and combination techniques, classification accuracy is improved with this multi-modal system when compared to the performance of each individual data source.

Figures (5)

• Figure 1: Example of ECoG Data
• Figure 2: Example of Piezo Data
• Figure 3: Example Frame of Video
• Figure 4: Comparison of pre- and post-FFT.
• Figure 5: VideoMAE Model Diagram b21