GelSphere: An Omnidirectional Rolling Vision-Based Tactile Sensor for Online 3D Reconstruction and Normal Force Estimation

Abstract

We present GelSphere, a spherical vision-based tactile sensor designed for real-time continuous surface scanning. Unlike traditional vision-based tactile sensors that can only sense locally and are damaged when slid across surfaces, and cylindrical tactile sensors that can only roll along a fixed direction, our design enables omnidirectional rolling on surfaces. We accomplish this through our novel sensing system design, which has steel balls inside the sensor, forming a bearing layer between the gel and the rigid housing that allows rolling motion in all axes. The sensor streams tactile images through Wi-Fi, with online large-surface reconstruction capabilities. We present quantitative results for both reconstruction accuracy and image fusion performance. The results show that our sensor maintains geometric fidelity and high reconstruction accuracy even under multi-directional rolling, enabling uninterrupted surface scanning.

Figures (7)

• Figure 1: Our novel vision-based tactile sensor, GelSphere, achieves omnidirectional continuous scanning of large surfaces through a unique mechanical design. It can be easily human-operated for online surface reconstruction or potentially used in robotic platforms for automation. We use tactile SLAM methods to stitch the contact patches together, forming a continuous surface model.
• Figure 2: Optical and mechanical design of GelSphere. (A) The simulated tactile image is shown alongside a real sensor image under a spherical indentation, presenting a close visual similarities between the optical simulation and the fabricated sensor. The RGB LED ring consists of 18 LEDs (6 of each color), and three color groups are spaced by approximately $120^\circ$, which provides directional illumination from every direction. (B) Cross-sectional view of the GelSphere sensor. The internal optical module (camera, LED right, and battery) is mounted on a base that is magnetically suspended, which helps maintain a consistent camera orientation while the outer shell rolls. A magnetic switch enables power on/off control without opening the housing. The steel ball bearing layer ensures low-friction contact between the sensing gel assembly and the housing, enabling smooth rolling during scanning. The sensor is self-contained, with onboard power and wireless image streaming.
• Figure 3: GelSphere sensor design. (A) Fabricated GelSphere components. The sensing gel pad is removable to allow battery replacement and maintenance. The cap is fastened to the housing to support the magnetic suspension structure and retain the steel balls in the bearing race. (B) Assembled prototype. The sensor is able to roll omnidirectionally when fully assembled.
• Figure 4: Fabrication Process for making the gel pad and the shell. The silicone mold (Mold Star 15) is created using the gel pad replica, and XP-565 silicone elastomer is added to the cured silicone mold. The off-the-shelf plastic shell is added to the top mold and pressed carefully with the bottom part of the mold, ensuring no air is caught. The cured elastomer is air-sprayed with the desired coating material to finish the gel pad.
• Figure 5: GelSphere's single frame sensing performance. (A) A 2D normal-estimation accuracy plot was generated from the indentation of a hexagonal pyramid at multiple locations. The RGB direction labels in the dot-product maps are for visual reference. The three LED groups are positioned $120^\circ$ apart around the ring. Left: specular, Right: matte. (B) A key and an M8 screw were pressed into both specular and matte sensing gel, showcasing the reconstruction performance.