Deep Reinforcement Learning-Based Semi-Autonomous Control for Magnetic Micro-robot Navigation with Immersive Manipulation

TL;DR

This work presents a deep reinforcement learning–based semi-autonomous control framework that integrates Mixed Reality–driven immersive manipulation with PPO-based autonomous navigation for magnetic micro-robots in simulated microvascular settings. By combining a hardware-focused platform, a digital twin–assisted immersive teleoperation loop, and a grid-world DRL policy, the approach enables seamless switching between manual and autonomous control to improve navigation efficiency, reduce errors, and lower operator workload. Key contributions include a digital twin–enabled MR manipulation system, a DRL-SC framework validated in simulation and transfer-tested on real hardware, and a user-study demonstrating faster navigation, fewer collisions, and reduced workload compared with manual control. The methodology has direct implications for safer, more efficient non-invasive microrobotic interventions, with future work aimed at extending to 3D microfluidic environments and in vitro biofluidic testing.

Abstract

Magnetic micro-robots have demonstrated immense potential in biomedical applications, such as in vivo drug delivery, non-invasive diagnostics, and cell-based therapies, owing to their precise maneuverability and small size. However, current micromanipulation techniques often rely solely on a two-dimensional (2D) microscopic view as sensory feedback, while traditional control interfaces do not provide an intuitive manner for operators to manipulate micro-robots. These limitations increase the cognitive load on operators, who must interpret limited feedback and translate it into effective control actions. To address these challenges, we propose a Deep Reinforcement Learning-Based Semi-Autonomous Control (DRL-SC) framework for magnetic micro-robot navigation in a simulated microvascular system. Our framework integrates Mixed Reality (MR) to facilitate immersive manipulation of micro-robots, thereby enhancing situational awareness and control precision. Simulation and experimental results demonstrate that our approach significantly improves navigation efficiency, reduces control errors, and enhances the overall robustness of the system in simulated microvascular environments.

Figures (8)

• Figure 1: Schematic of the Deep Reinforcement Learning-Based Semi-Autonomous Control (DRL-SC) framework for magnetic micro-robot navigation with Mixed Reality (MR)-based immersive manipulation. The diagram illustrates the key components of the framework, including MR-based manual control, DRL-based autonomous control, and the physical environment for magnetic micro-robot navigation.
• Figure 2: Overview of the experimental setup: This system includes a micromanipulation with a magnetic microneedle for non-contact manipulation of the magnetic micro-robot, a digital microscope for acquiring high-resolution 2D microscopic views, and a magnified visualization of the simulated microvascular environment.
• Figure 3: The model learning curve of PPO and A2C.
• Figure 4: (a), (b), and (c) each show three sets of experiments with randomly selected starting and ending points, illustrating the navigation performance of the PPO model in both virtual and real environments.
• Figure 5: The experimental scenario demonstrates teleoperation for navigating a magnetic micro-robot using MR-based manual control.