Soft Surfaced Vision-Based Tactile Sensing for Bipedal Robot Applications

TL;DR

This paper addresses stability and environmental awareness for bipedal robots by embedding a soft, vision-based tactile sensor in each foot to directly sense foot-ground contact. A data-driven perception pipeline reconstructs depth, CoP, shear, object pose, and terrain type from contact images, enabling real-time closed-loop balance on a tilting platform and terrain classification under visual occlusion. Key contributions include the foot-mounted GelSight-like sensor design, a multi-task perception stack with a depth net and Poisson reconstruction, shear estimation from marker grids, and validated gains in balance performance and terrain awareness. The approach advances compliant, adaptive locomotion by turning tactile feedback into actionable control signals for bipedal robots in unstructured or partially visible environments.

Abstract

Legged locomotion benefits from embodied sensing, where perception emerges from the physical interaction between body and environment. We present a soft-surfaced, vision-based tactile foot sensor that endows a bipedal robot with a skin-like deformable layer that captures contact deformations optically, turning foot-ground interactions into rich haptic signals. From a contact image stream, our method estimates contact pose (position and orientation), visualizes shear, computes center of pressure (CoP), classifies terrain, and detects geometric features of the contact patch. We validate these capabilities on a tilting platform and in visually obscured conditions, showing that foot-borne tactile feedback improves balance control and terrain awareness beyond proprioception alone. These findings suggest that integrating tactile perception into legged robot feet improves stability, adaptability, and environmental awareness, offering a promising direction toward more compliant and intelligent locomotion systems. For the supplementary video, please visit: https://youtu.be/ceJiy9q_2Aw

Figures (11)

• Figure 1: Foot sensor design. (a) Isometric view of the sensor. (b) Side view with partially removed housing. (c) Exploded view with labeled parts.
• Figure 2: Depth reconstruction from the sensor using a trained model.
• Figure 3: Detecting shear force with dotted grid pad. Shear force is represented with pink and red arrows.
• Figure 4: Detecting position and orientation of the contact region of different geometries. Raw images in the upper row, detected images in the lower row. (a) Corner of a cube. (b) Circular edge of a cylinder. (c) Edge of a prism. (d) Screw head.
• Figure 5: Processed image with OpenCV function. The image processing steps flow from left to right to detect where the pressure is being applied.