Maglev-Pentabot: Magnetic Levitation System for Non-Contact Manipulation using Deep Reinforcement Learning

TL;DR

This work tackles macroscopic non-contact manipulation by magnetic levitation and introduces Maglev-Pentabot, a vision-guided system powered by deep reinforcement learning. It combines an optimized multi-magnet execution module, a DRL-based controller, and an action remapping strategy to cope with nonlinear magnetic fields, enabling fast and precise levitation and transportation of gram-scale objects. The approach demonstrates 2D and 3D levitation and load-transport tasks, including generalization to unseen payloads, and provides scalability analysis suggesting industrial-scale applications with larger magnets. The study also analyzes convergence differences between PPO and SAC in high-dimensional maglev control and discusses hardware limitations and mitigation strategies, offering a reference framework for scalable, non-contact manipulation robotics.

Abstract

Non-contact manipulation has emerged as a transformative approach across various industrial fields. However, current flexible 2D and 3D non-contact manipulation techniques are often limited to microscopic scales, typically controlling objects in the milligram range. In this paper, we present a magnetic levitation system, termed Maglev-Pentabot, designed to address this limitation. The Maglev-Pentabot leverages deep reinforcement learning (DRL) to develop complex control strategies for manipulating objects in the gram range. Specifically, we propose an electromagnet arrangement optimized through numerical analysis to maximize controllable space. Additionally, an action remapping method is introduced to address sample sparsity issues caused by the strong nonlinearity in magnetic field intensity, hence allowing the DRL controller to converge. Experimental results demonstrate flexible manipulation capabilities, and notably, our system can generalize to transport tasks it has not been explicitly trained for. Furthermore, our approach can be scaled to manipulate heavier objects using larger electromagnets, offering a reference framework for industrial-scale robotic applications.

Figures (10)

• Figure 1: Diagram of Maglev-Pentabot. The Maglev-Pentabot leverages a Deep Reinforcement Learning (DRL) controller as its brain, camera as its vision, and electromagnets coupled with a magnetized ball as the execution module. First, actuator motion information is obtained through the camera. Then, the DRL controller infers control action by inputting this information, thereby achieving control of the actuator.
• Figure 2: Framework of Maglev-Pentabot. The workflow proceeds as follows: 1. The camera captures images of the actuator's (magnetized ball's) movement and transmits them to the Upper Computer. 2. On the Upper Computer, the motion information of the actuator (such as position, velocity, and acceleration) are extracted through image analysis. The DRL controller then uses this information (as the observation of DRL controller) to infer control action, which are sent to the Arduino on Lower Computer. 3. The Arduino converts the control action into PWM (Pulse Width Modulation) signal and, via the drive module, converts these PWM signal into drive signal that control the current in the electromagnets (thus controlling the magnetic field strength). This governs the actuator's movement, achieving precise motion control of the magnetized ball.
• Figure 3: Controllable area in 2D scenario. (A) Magnets with same polarities. (B) Magnets with different polarities.
• Figure 4: Execution module in 2D and 3D scenarios. (A) Diagram of 2D Execution module. (B) Prototype of 2D Execution module. (C) Camera's field of view in 2D scenario. (D) Diagram of 3D Execution module. (E) Prototype of 3D Execution module. (F) Camera's field of view in 3D scenario.
• Figure 5: Network structures of DRL agents. (A) The structure of PPO. (B) The structure of SAC.