In-Vivo Training for Deep Brain Stimulation

TL;DR

This work presents an in-vivo trainable closed-loop DBS framework using a TD3 reinforcement learning agent trained on a brain-on-chip BG–thalamus (BGT) model. The state space integrates in-vivo–measurable biomarkers, including $S_{Gi}$ PSD and GPi beta-band activity, with Hjorth parameters and STN entropy, while the reward combines PD biomarker suppression and energy efficiency via $r_1$ and $r_2$ and a weighted sum $r=\epsilon(-r_1)+(1-\epsilon)(-r_2)$ using $\epsilon=0.68$ and $\theta=0.85$. The TD3-DBS agent achieves a $7.35\%$ reduction in $S_{Gi}$ PSD and a $6.93\%$ reduction in $V_{Gi}$ PSD relative to open-loop DBS, with about a $31\%$ decrease in power usage, and operates with a mean stimulation frequency near $135$ Hz and lower amplitude, illustrating improved biomarker control with greater energy efficiency. These results demonstrate the feasibility of in-vivo–training for personalized, safe, and efficient closed-loop DBS, potentially enabling real-time deployment (e.g., on FPGA) with safety safeguards.

Abstract

Deep Brain Stimulation (DBS) is a highly effective treatment for Parkinson's Disease (PD). Recent research uses reinforcement learning (RL) for DBS, with RL agents modulating the stimulation frequency and amplitude. But, these models rely on biomarkers that are not measurable in patients and are only present in brain-on-chip (BoC) simulations. In this work, we present an RL-based DBS approach that adapts these stimulation parameters according to brain activity measurable in vivo. Using a TD3 based RL agent trained on a model of the basal ganglia region of the brain, we see a greater suppression of biomarkers correlated with PD severity compared to modern clinical DBS implementations. Our agent outperforms the standard clinical approaches in suppressing PD biomarkers while relying on information that can be measured in a real world environment, thereby opening up the possibility of training personalized RL agents specific to individual patient needs.

Figures (1)

• Figure 1: The RL algorithm's interaction with the BGT model. The reward term is only used during training (dotted line).