Model-Guided Microstimulation Steers Primate Visual Behavior

TL;DR

This paper addresses the challenge of eliciting complex object-level visual percepts by directly stimulating higher-level visual cortex. It introduces a model-guided framework that couples topographic deep networks (TDANNs) with a perturbation module and a model-to-brain mapping, enabling in-silico prototyping and in-vivo testing in macaques performing object recognition tasks. Across two subjects, model predictions correlated with observed behavioral shifts (approximately $r \approx 0.53$–$0.58$) and, in a second experiment, produced a significant perceptual bias, with GAN-based visualizations showing face-like percepts when stimulating face-selective cortex. This work provides a principled foundation for prosthetics aimed at evoking richer, object-level visual experiences and suggests broader utility for model-guided causal interventions beyond vision.

Abstract

Brain stimulation is a powerful tool for understanding cortical function and holds promise for therapeutic interventions in neuropsychiatric disorders. Initial visual prosthetics apply electric microstimulation to early visual cortex which can evoke percepts of simple symbols such as letters. However, these approaches are fundamentally limited by hardware constraints and the low-level representational properties of this cortical region. In contrast, higher-level visual areas encode more complex object representations and therefore constitute a promising target for stimulation - but determining representational targets that reliably evoke object-level percepts constitutes a major challenge. We here introduce a computational framework to causally model and guide stimulation of high-level cortex, comprising three key components: (1) a perturbation module that translates microstimulation parameters into spatial changes to neural activity, (2) topographic models that capture the spatial organization of cortical neurons and thus enable prototyping of stimulation experiments, and (3) a mapping procedure that links model-optimized stimulation sites back to primate cortex. Applying this framework in two macaque monkeys performing a visual recognition task, model-predicted stimulation experiments produced significant in-vivo changes in perceptual choices. Per-site model predictions and monkey behavior were strongly correlated, underscoring the promise of model-guided stimulation. Image generation further revealed a qualitative similarity between in-silico stimulation of face-selective sites and a patient's report of facephenes. This proof-of-principle establishes a foundation for model-guided microstimulation and points toward next-generation visual prosthetics capable of inducing more complex visual experiences.

Figures (15)

• Figure 1: Overview of approach. Existing approaches to visual prosthetics microstimulate early cortical areas or even earlier parts of the visual processing hierarchy, do not use computational models for the selection of stimulation sites, and instead rely on retinotopic organization where nearby locations in the visual field are represented in nearby locations in the early visual system. These approaches have successfully been shown to elicit percepts of simple visual symbols such as letters, but are limited by the low-level representational properties of early visual regions. We propose a model-guided approach that targets higher-level visual cortex via computational simulations with the goal of eliciting percepts of complex visual objects.
• Figure 2: Model-guided microstimulation.1. Model–brain mapping. To align model tissue and monkey brain recordings, we use passive-viewing responses of 4,000 images recorded 2–4 days before each experimental session. We then simulate various positionings of an electrode grid on the topographic tissue of model candidates, selecting the model grid position and orientation that maximizes correlations between model and monkey recording sites. This yields a fixed one-to-one mapping between sites in the model and brain-implanted electrode grid. 2. Prototype experiments in model. For each candidate site we generate sequences of seven images varying smoothly along GAN latent space, rank them by a selectivity score (slope-to-noise), and test the effect of microstimulation on model-predicted 2AFC behavioral choices. Deepest-layer representations are converted to two-alternatives-forced-choice responses via similarity comparisons. 3. Test model-selected parameters in primate. We select the top site–sequence predictions, mapping model neural sites back to the corresponding IT electrodes, and deploy the monkey experiment in a 2AFC recognition task. Biphasic trains of electric stimulation are delivered on designated trials, interleaved with sham. Full details in Appendix Sec. \ref{['sec:experimental_setup_monkey']}.
• Figure 3: Model predictions correlate with stimulation-evoked behavioral shifts.A) Model-predicted behavioral shifts ($\Delta$AUC) correlate with stimulation-evoked shifts in the monkeys’ behavioral responses ($\Delta$AUC), both when combining across the two subjects (Pearson $r = 0.53$, $p = 0.0012$). B) Example psychometric functions from two stimulation sites (gray symbols in A).
• Figure 4: Model-guided stimulation biases monkey behavior.A) In experiment 2, model-guided stimulation induced a significant behavioral shift in monkey 1 (Wilcoxon signed rank test: $p = 0.043$; Cohen’s $d = 0.67$). Due to declining signal quality, experiment 2 could not be conducted with monkey 2. B) Three example GAN-generated image sequences used for stimulation (images 1, 4, and 7 shown from each seven-image sequence; corresponding sites highlighted in A).
• Figure 5: Visualizing perceptual effects of microstimulation. Two examples of simulated stimulation effects in the model. Simulated current amplitude increases from left ($0\,\mu\text{A}$) to right ($1000\,\mu\text{A}$). In the first row, stimulation transforms a cat's tail into an additional face, while in the second row it enlarges the face of a bear.