OmniNeuro: A Multimodal HCI Framework for Explainable BCI Feedback via Generative AI and Sonification

TL;DR

OmniNeuro tackles the clinical bottleneck of opaque brain-computer interfaces by replacing a silent decoder with a transparent, multimodal feedback system. It introduces a decoder-agnostic, white-box architecture built on three interpretability engines—Physics (energy), Chaos (complexity), and Quantum-inspired uncertainty—driving real-time Neuro-Sonification and AI-generated clinical reports. Although decoding accuracy remains comparable to baselines, the feedback-centric design enhances learning stability and user satisfaction, as shown by quantitative improvements and a qualitative pilot with three participants. The framework supports closed-loop neurofeedback, potentially improving neuroplasticity in rehabilitation and offering a practical bridge for clinical adoption through interpretable, actionable feedback rather than raw performance alone.

Abstract

While Deep Learning has improved Brain-Computer Interface (BCI) decoding accuracy, clinical adoption is hindered by the "Black Box" nature of these algorithms, leading to user frustration and poor neuroplasticity outcomes. We propose OmniNeuro, a novel HCI framework that transforms the BCI from a silent decoder into a transparent feedback partner. OmniNeuro integrates three interpretability engines: (1) Physics (Energy), (2) Chaos (Fractal Complexity), and (3) Quantum-Inspired uncertainty modeling. These metrics drive real-time Neuro-Sonification and Generative AI Clinical Reports. Evaluated on the PhysioNet dataset ($N=109$), the system achieved a mean accuracy of 58.52%, with qualitative pilot studies ($N=3$) confirming that explainable feedback helps users regulate mental effort and reduces the "trial-and-error" phase. OmniNeuro is decoder-agnostic, acting as an essential interpretability layer for any state-of-the-art architecture.

Figures (7)

• Figure 1: Conceptual Framework: From Command Decoding to Closed-Loop Neurofeedback. While standard decoders (top row) focus on external control, OmniNeuro (bottom loop) focuses on internal state regulation, creating a stabilizing feedback loop essential for neuroplasticity.
• Figure 2: The OmniNeuro HCI Architecture. Scaled to fit within page margins.
• Figure 3: Quantum-Inspired Representation. This visualization allows users to see their "mental vector" drifting towards the target state.
• Figure 4: Subject-wise Performance. OmniNeuro (y-axis) vs Baseline (x-axis). The system provides a safety net of explainability even for lower-accuracy subjects.
• Figure 5: Integrated System Output: Real-time console logs showing the transparency of the decision-making process.