Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neurophysiologically Realistic Environment for Comparing Adaptive Deep Brain Stimulation Algorithms in Parkinson Disease

Ekaterina Kuzmina, Dmitrii Kriukov, Mikhail Lebedev, Dmitry V. Dylov

TL;DR

This work addresses the challenge of validating adaptive DBS (aDBS) policies for Parkinson disease by introducing a neurophysiologically realistic benchmark built on Kuramoto oscillator dynamics. The environment encodes bandwidth, spatial, and temporal features, including beta-band activity, electrode-drift, neural plasticity, and directional stimulation, to train and evaluate reinforcement learning controllers in a controlled, configurable setting. It demonstrates how online RL methods, particularly SAC, can learn strategies that suppress pathological beta oscillations while managing energy expenditure across progressively complex environments, highlighting the value of a standardized pretraining platform for ML-driven neurostimulation. The framework aims to improve generalizability and stability of aDBS strategies and to accelerate their translation by providing an open, scalable benchmark for ML researchers and clinicians alike.

Abstract

Adaptive deep brain stimulation (aDBS) has emerged as a promising treatment for Parkinson disease (PD). In aDBS, a surgically placed electrode sends dynamically altered stimuli to the brain based on neurophysiological feedback: an invasive gadget that limits the amount of data one could collect for optimizing the control offline. As a consequence, a plethora of synthetic models of PD and those of the control algorithms have been proposed. Herein, we introduce the first neurophysiologically realistic benchmark for comparing said models. Specifically, our methodology covers not only conventional basal ganglia circuit dynamics and pathological oscillations, but also captures 15 previously dismissed physiological attributes, such as signal instabilities and noise, neural drift, electrode conductance changes and individual variability - all modeled as spatially distributed and temporally registered features via beta-band activity in the brain and a feedback. Furthermore, we purposely built our framework as a structured environment for training and evaluating deep reinforcement learning (RL) algorithms, opening new possibilities for optimizing aDBS control strategies and inviting the machine learning community to contribute to the emerging field of intelligent neurostimulation interfaces.

Neurophysiologically Realistic Environment for Comparing Adaptive Deep Brain Stimulation Algorithms in Parkinson Disease

TL;DR

This work addresses the challenge of validating adaptive DBS (aDBS) policies for Parkinson disease by introducing a neurophysiologically realistic benchmark built on Kuramoto oscillator dynamics. The environment encodes bandwidth, spatial, and temporal features, including beta-band activity, electrode-drift, neural plasticity, and directional stimulation, to train and evaluate reinforcement learning controllers in a controlled, configurable setting. It demonstrates how online RL methods, particularly SAC, can learn strategies that suppress pathological beta oscillations while managing energy expenditure across progressively complex environments, highlighting the value of a standardized pretraining platform for ML-driven neurostimulation. The framework aims to improve generalizability and stability of aDBS strategies and to accelerate their translation by providing an open, scalable benchmark for ML researchers and clinicians alike.

Abstract

Adaptive deep brain stimulation (aDBS) has emerged as a promising treatment for Parkinson disease (PD). In aDBS, a surgically placed electrode sends dynamically altered stimuli to the brain based on neurophysiological feedback: an invasive gadget that limits the amount of data one could collect for optimizing the control offline. As a consequence, a plethora of synthetic models of PD and those of the control algorithms have been proposed. Herein, we introduce the first neurophysiologically realistic benchmark for comparing said models. Specifically, our methodology covers not only conventional basal ganglia circuit dynamics and pathological oscillations, but also captures 15 previously dismissed physiological attributes, such as signal instabilities and noise, neural drift, electrode conductance changes and individual variability - all modeled as spatially distributed and temporally registered features via beta-band activity in the brain and a feedback. Furthermore, we purposely built our framework as a structured environment for training and evaluating deep reinforcement learning (RL) algorithms, opening new possibilities for optimizing aDBS control strategies and inviting the machine learning community to contribute to the emerging field of intelligent neurostimulation interfaces.
Paper Structure (39 sections, 7 equations, 8 figures, 3 tables)

This paper contains 39 sections, 7 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. PD models for aDBS training
  4. Existing algorithms for aDBS
  5. Problem setup
  6. Neural dynamics model description
  7. Features description and justification
  8. Bandwidth features.
  9. Spatial features.
  10. Temporal features
  11. DBS control problem
  12. Observation
  13. Action space
  14. Evaluation metrics and reward function.
  15. Complexity levels of proposed environment
  16. ...and 24 more sections

Figures (8)

  • Figure 1: Proposed Environment. A) A closed-loop electrical brain stimulation system measures neural activity through local field potential (LFP, 'Observation') to compute the needed stimuli ('Action') in real time. Then, a chosen algorithm (e.g., RL) should learn to control any undesired signaling in the simulated brain model ('Environment'). B) Three groups of features of the proposed brain model. Bandwidth features cover fundamental neurobiological activity as beta oscillations and bursts. Spatial features describe spatial relationships between the electrode and surrounding neurons. Temporal features introduce experimentally relevant noise into learning, simulating neuroplasticity and glial encapsulation and requiring control resilient to the environmental drift. The Kuramoto equations map the corresponding coefficients (color-coded) to the feature groups. C) An example of LFP dynamics with DBS off and on (a high-frequency HF-DBS with a constant stimulation amplitude is shown). D) A comparison of beta burst distribution observed in real patients with PD (taken from tinkhauser2017beta) and those simulated by the proposed model. E) The power spectral density of LFP signals from (C). Note the suppression of beta-band power with constant HF-DBS. F) Modern synthetic Parkinson's disease models and the features they cover (indicated with filled squares).
  • Figure 2: Spatial and phase dynamics of LFP for different placement of the multi-contact electrode relative to beta locus (environment parameters in Table \ref{['tab:configs']}). A) Beta locus at the center of the signaling neural grid. B) The corresponding phase portrait, shown for no DBS (black) and HF-DBS stimulation (orange). C) The beta locus at the corner. D) The corner-based phase portrait shows decreased synchronization of neurons (note smaller cycle amplitude).
  • Figure 3: Performance of selected DBS Algorithms in proposed suppression stability task: A) SAC-DBS and B) DDPG-DBS. The left column shows beta oscillation suppression (black: No DBS, orange: DBS), with red markers indicating DBS pulse amplitudes. The second column depicts the DBS amplitude distribution. The third column displays LFP power spectral densities, highlighting beta-band suppression. The right column illustrates control stability performance mimicking real-life 'experimental' issues encoded as an event sequence: electrode movement (diamonds, simulating random shifts in the observation position), neural drift (squares, reflecting changes in the natural frequencies of neurons), encapsulation (hexagons, representing degradation in conductance between the DBS electrode and surrounding neurons). Beta power per episode (black and orange) and total stimulation energy (red points) are also shown.
  • Figure A1: Performance of Different Control Strategies in three proposed Environments. The top panels show the efficiency of beta-band power suppression, expressed as a percentage relative to the average beta-band power in the uncontrolled environment (DBS OFF case, represented by the grey dashed line). The bottom panels display the energy consumption efficiency of different control strategies, measured as a percentage relative to the case of maximum energy consumption (as in the HF-DBS ON condition, indicated by the orange dashed line). Columns A, B, and C correspond to environment levels 0, 1, and 2, each incorporating different feature groups. The level 0 environment includes only bandwidth features, while level 1 adds spatial features. The level 2 environment incorporates features from all three groups. Error bars represent one standard deviation across 10 environment runs for env0 and env1 and 25 for env2 conducted during the validation procedure.
  • Figure A2: Phase portraits of LFP dynamics at different beta locus positions relative to the electrode for different coupling parameters and the number of neurons in the grid. A) The case for 512 neurons in the grid. Left, sketches demonstrate a position of beta locus in the neural grid. Right, the corresponding phase portraits of LFPs at different coupling parameters. B) The same is also shown for the case of 1331 neurons in the grid.
  • ...and 3 more figures