MagicGel: A Novel Visual-Based Tactile Sensor Design with MagneticGel

TL;DR

MagicGel addresses the limitations of vision-based tactile sensors by integrating a magnetic perception layer with Hall sensors into a VBTS, enabling both high-fidelity force estimation and non-contact proximity sensing. The approach fuses visual features from a CNN with time-series magnetic features from a GRU, concatenating them for regression to predict normal force, achieving an RMSE of 0.0497 N. Key contributions include a novel Vision-Magnetic fusion tactile sensor design, a CNN-GRU fusion framework, and experimental validation showing improved force estimation accuracy and proximity perception. The resulting multimodal sensor offers faster feedback and enhanced dexterous manipulation capabilities in robotics.

Abstract

Force estimation is the core indicator for evaluating the performance of tactile sensors, and it is also the key technical path to achieve precise force feedback mechanisms. This study proposes a design method for a visual tactile sensor (VBTS) that integrates a magnetic perception mechanism, and develops a new tactile sensor called MagicGel. The sensor uses strong magnetic particles as markers and captures magnetic field changes in real time through Hall sensors. On this basis, MagicGel achieves the coordinated optimization of multimodal perception capabilities: it not only has fast response characteristics, but also can perceive non-contact status information of home electronic products. Specifically, MagicGel simultaneously analyzes the visual characteristics of magnetic particles and the multimodal data of changes in magnetic field intensity, ultimately improving force estimation capabilities.

Figures (6)

• Figure 1: MagicGe structure diagram. (1) Elastomer 1. (2) Magnetic particle marking points. (3) Light source (4) Elastomer 2. (5) Support plate. (6) Hall sensor circuit board. (7) Connectors. (8) Base. (9) Camera.
• Figure 2: MagicGel manufacturing process flow chart. In the figure, (1) and (2) are molds 1 and 2, and (3) and (4) are magnets 1 and 2, (5) is transparent light source isolation housing. (a) Silicone material. (b) Surface spray material manufacturing process and spray quality (microscopic). The eight Hall sensors in Assemble 2 are evenly integrated into the circuit board.
• Figure 3: Schematic diagram of the network model for force measurement using VBTS. The RGB three-channel information of the tactile image and the 2D information of the magnetic tactile image are spliced and integrated to achieve the fusion of visual and magnetic information.
• Figure 4: (1) Dynamometer(Standard force, Move horizontally, press vertically). (2) Pressure head. (3) MagicGel. (4) Magnetic Data Module. (5) Visual data receiving module. Model 1, model 2 and model 3 are models trained on three different sets of data.
• Figure 5: Force measurement results (where a, b, c are force/N, and d is the data number). (a)(b)(c)Magnetic tactile, Visual tactile force and Visual-magnetic fusion measurement results. (d) Visual-Magnetic force estimation verification result line chart. True value (red), estimated value (green). The degree of dispersion of the points in the graph represents the degree of expressiveness estimation.