A Multi-Camera Optical Tag Neuronavigation and AR Augmentation Framework for Non-Invasive Brain Stimulation

TL;DR

This work tackles the high cost and usability barriers of neuronavigation for transcranial magnetic stimulation by introducing a low-cost, optical tag-based, multi-camera tracking system synchronized with an AR visualization pipeline. Using printed AprilTag fiducials and a trio of consumer cameras, the method achieves real-time 6 DoF tracking of the TMS coil and patient head, fused across views for robustness, and maps positions into a Unity-based brain model with AR overlays. The system delivers sub-pixel 2D reprojection accuracy (~0.063–0.069 px), depth precision of 0.07–0.09 mm, and angular precision of 0.04–0.06°, with millimeter-scale target localization, while maintaining interactive latency (~0.59 s). A case study with ten medically oriented participants demonstrates high usability and practical potential, suggesting that this affordable framework can make precise TMS neuronavigation more accessible in diverse settings.

Abstract

Accurate neuronavigation is essential for generating the intended effect with transcranial magnetic stimulation (TMS). Precise coil placement also directly influences stimulation efficacy. Traditional neuronavigation systems often rely on costly and still hard to use and error-prone tracking systems. To solve these limitations, we present a computer-vision-based neuronavigation system for real-time tracking of patient and TMS instrumentation. The system can feed the necessary data for a digital twin to track TMS stimulation targets. We integrate a self-coordinating optical tracking system with multiple consumer-grade cameras and visible tags with a dynamic 3D brain model in Unity. This model updates in real time to represent the current stimulation coil position and the estimated stimulation point to intuitively visualize neural targets for clinicians. We incorporate an augmented reality (AR) module to bridge the gap between the visualization of the digital twin and the real world and project the brain model in real-time onto the head of a patient. AR headsets or mobile AR devices allow clinicians to interactively view and adjust the placement of the stimulation transducer intuitively instead of guidance through abstract numbers and 6D cross hairs on an external screen. The proposed technique provides improved spatial precision as well as accuracy. A case study with ten participants with a medical background also demonstrates that the system has high usability.

Figures (14)

• Figure 1: Illustration of TMS treatment. The TMS coil is placed on the patient’s head with millimeter accuracy above a specific neural target to target a certain circuit, e.g., often a control loop, for clinical therapy or brain research.
• Figure 2: The optical tag-based neuronavigation system combines multi-camera tracking with augmented reality (AR) visualization to improve transcranial magnetic stimulation (TMS) procedures taken from hu2026opticaltagbasedneuronavigationaugmentation. This system uses low-cost web cameras to track black-and-white optical fiducials on the TMS coil and patient for real-time tracking. Unity 3D visualization and AR overlays guide precise coil positioning and target identification.
• Figure 3: Representation of the planar black-and-white tag family (AprilTag tag36h11). The tag's four corner points are labeled for accurate pose estimation. A coordinate frame at the tag center indicates its orientation and position in space, which can be derived from the corner positions even from a single camera perspective.
• Figure 4: Relationship between the image plane (2D pixel space), the camera coordinate system (3D space), and the world coordinate system (anchored to the patient via an fiducial marker). These transformations allow accurate localization of the TMS coil in the patient's head frame for real-time AR visualization and navigation.
• Figure 5: (a) Components and (b) multi-camera TMS neuronavigation system during experiment taken from hu2026opticaltagbasedneuronavigationaugmentation. The system consists of a TMS coil, a multi-camera tracking setup for real-time spatial localization, and flat optical markers attached to the tracked objects and the head for precise position and orientation detection. The cameras provide synchronized tracking data and translate all components into a unified coordinate frame.