MRUCT: Mixed Reality Assistance for Acupuncture Guided by Ultrasonic Computed Tomography

TL;DR

MRUCT addresses the lack of imaging guidance in acupuncture by fusing Ultrasonic Computed Tomography (UCT) with Mixed Reality (MR) to visualize bones and muscles in situ and to overlay auto-generated insertion trajectories. It relies on offline non-rigid registration via Large Deformation Diffeomorphic Metric Mapping (LDDMM) to align template anatomy with patient anatomy, producing reference trajectories that can be adjusted by practitioners in real time. An attention-adaptive 3DUI on the HoloLens guides ultrathin needle insertion, supported by robust tracking and a detachable needle tracker, achieving superior usability and insertion accuracy compared with traditional methods and a two-ring UI. The study demonstrates MRUCT’s potential to enhance acupuncture training and clinical practice, with quantified improvements in registration accuracy, end-to-end system error (~0.32 mm), and user experience, while outlining future work on broader validation, force feedback, and real-time tissue deformation modeling.

Abstract

Chinese acupuncture practitioners primarily depend on muscle memory and tactile feedback to insert needles and accurately target acupuncture points, as the current workflow lacks imaging modalities and visual aids. Consequently, new practitioners often learn through trial and error, requiring years of experience to become proficient and earn the trust of patients. Medical students face similar challenges in mastering this skill. To address these challenges, we developed an innovative system, MRUCT, that integrates ultrasonic computed tomography (UCT) with mixed reality (MR) technology to visualize acupuncture points in real-time. This system offers offline image registration and real-time guidance during needle insertion, enabling them to accurately position needles based on anatomical structures such as bones, muscles, and auto-generated reference points, with the potential for clinical implementation. In this paper, we outline the non-rigid registration methods used to reconstruct anatomical structures from UCT data, as well as the key design considerations of the MR system. We evaluated two different 3D user interface (3DUI) designs and compared the performance of our system to traditional workflows for both new practitioners and medical students. The results highlight the potential of MR to enhance therapeutic medical practices and demonstrate the effectiveness of the system we developed.

Figures (9)

• Figure 1: Design cards illustrating different prototype design choices were used, with a short description of the system workflow attached to each card to facilitate understanding and discussion. Note that the two-ring UI and 2D ultrasound images were adopted from previous studies hzhang2024strighttrackleow2017exploring.
• Figure 2: Schematic diagram of data processing workflow.
• Figure 3: The transformation system of MRUCT. The HMD sensor tracks the patient's arm and needle and provides real-time transformation. The 3D imaging reconstruction is aligned to the patient's arm through local coordinates of retro-reflective mark coordinates and four labeled mesh coordinates (yellow triangles). The reference insertion trajectory is automatically computed from entry and end points (red circles).
• Figure 4: The identification of local coordinates of marker centers and four mesh in imaging space. The coordinates were used to spatially align the 3D reconstructed mesh to the patient's arm position.
• Figure 5: Illustration of MRUCT attention-adaptive 3DUI (a,b,c) and the two ring system (d). (a) shows the mechanism of the tip indicator navigation system. (b) illustrates the activation of the rotation error indicator. (c) demonstrates a precisely placed needle by using MRUCT. (d) is the comparative method