MindPilot: Closed-loop Visual Stimulation Optimization for Brain Modulation with EEG-guided Diffusion

TL;DR

MindPilot addresses the challenge of non-invasively modulating brain states by using EEG-derived targets to guide diffusion-based image generation in a closed-loop, gradient-free framework. It treats the brain as a non-differentiable black-box $g$ and leverages a surrogate pseudo-model to provide gradient-like signals for optimization, enabling semantic and spectral target alignment. The approach combines interactive search, GP-based guidance, and diffusion generation to produce naturalistic stimuli that steer neural representations toward target embeddings, validated in simulations and real-time human EEG experiments including emotion regulation tasks. The work demonstrates the feasibility of EEG-guided brain–stimulus co-optimization with potential implications for bidirectional BCIs, cognitive neuroscience, and non-invasive neuromodulation.

Abstract

Whereas most brain-computer interface research has focused on decoding neural signals into behavior or intent, the reverse challenge-using controlled stimuli to steer brain activity-remains far less understood, particularly in the visual domain. However, designing images that consistently elicit desired neural responses is difficult: subjective states lack clear quantitative measures, and EEG feedback is both noisy and non-differentiable. We introduce MindPilot, the first closed-loop framework that uses EEG signals as optimization feedback to guide naturalistic image generation. Unlike prior work limited to invasive settings or low-level flicker stimuli, MindPilot leverages non-invasive EEG with natural images, treating the brain as a black-box function and employing a pseudo-model guidance mechanism to iteratively refine images without requiring explicit rewards or gradients. We validate MindPilot in both simulation and human experiments, demonstrating (i) efficient retrieval of semantic targets, (ii) closed-loop optimization of EEG features, and (iii) human-subject validations in mental matching and emotion regulation tasks. Our results establish the feasibility of EEG-guided image synthesis and open new avenues for non-invasive closed-loop brain modulation, bidirectional brain-computer interfaces, and neural signal-guided generative modeling.

Figures (29)

• Figure 1: Conceptualization of MindPilot.A. The goal of MindPilot is to continuously optimize visual stimuli to drive the brain latent state to the target. B. A pseudo-model provides surrogate gradients to iteratively refine images with respect to neural targets (e.g., semantic feature, spectral feature). C. The closed-loop visual optimization.
• Figure 2: Framework. A. Each MindPilot iteration involves four steps: 1): The black-box proxy model $g$, which maps images to synthetic EEG, is designed as a black-box proxy model to predict the brain responses $\bm{X}$. 2): The EEG Encoder $f$ identifies different kinds of features $\bm{Y}$ from EEG. MindPilot calculates the similarity score $sim(f(g(\bm{u})), \bm{y}_\text{target})$ as rewards based on EEG features. 3): Update the image embedding using gradient descent. 4): The image with a higher brain similarity score is selected and passed back to the image generator to optimize stimuli. B. Semantic feature from a pre-trained EEG encoder $f$, aligned with CLIP embedding. C. Brain energy feature using Power Spectral Density (PSD) features. For more details, refer to Section \ref{['sec:method_framework']}.
• Figure 3: EEG-guided Interactive Search.A. Similarity score between MindPilot’s neural representations (steps 1, 2, last, and best-step) and the target, versus random stimuli. B. Cross-modal correlation between image and EEG embedding similarity ($R=0.23$, $P<0.01$). C. Similarity score improvement across all subjects using semantic features. D. The correlation between image embedding similarity and EEG semantic feature similarity across all subjects. The vertical axis represents the similarity score between the EEG features at the current step and the target.
• Figure 4: EEG semantic feature-guided visual stimuli generation.A. Mean L1 error reduction relative to the random stimuli baseline and offline EEG-guided generation, averaged over five closed-loop experimental targets. B. CLIP similarity scores for semantic reconstruction across three different estimators in the pseudo-model setup. C. Ablation study on optimization guidance modalities. Top: Original ground-truth target images. Middle (Upper Bound): Generation results optimized directly via target image features (CLIP). Bottom (Ours): Generation results optimized via EEG features (MindPilot), demonstrating the model's capability to bridge the modality gap under real-world noisy conditions.
• Figure 5: EEG PSD-driven generation.A. Neural-visual correlation analysis. We report the correlation coefficients between EEG PSD feature similarity and image CLIP feature similarity across 10 subjects in the THINGS-EEG2 dataset. B. Examples of heuristic generation guided by PSD feature across EEG channels. C. Improvement in feature similarity scores across all 10 subjects, with statistical significance determined by paired t-tests.