TaSA: Two-Phased Deep Predictive Learning of Tactile Sensory Attenuation for Improving In-Grasp Manipulation

TL;DR

TaSA addresses the challenge of discriminating self-generated tactile signals from object contact in dexterous, in-hand manipulation. It introduces a two-phased predictive framework: first learn a self-touch forward model from finger–finger and finger–palm contacts, then integrate a frozen copy of this model into a recurrent motion learner that attends to predicted self-touch and raw tactile input to isolate object interactions. Across three precision-insertion tasks, TaSA significantly improves success rates and generalization, demonstrating the importance of structured tactile perception with self-touch attenuation. The approach leverages high-resolution uSkin tactile sensing on the Allegro Hand and shows how predictive self-touch attenuation can stabilize manipulation in contact-rich scenarios, with PCA analyses confirming clearer decision boundaries under attenuation.

Abstract

Humans can achieve diverse in-hand manipulations, such as object pinching and tool use, which often involve simultaneous contact between the object and multiple fingers. This is still an open issue for robotic hands because such dexterous manipulation requires distinguishing between tactile sensations generated by their self-contact and those arising from external contact. Otherwise, object/robot breakage happens due to contacts/collisions. Indeed, most approaches ignore self-contact altogether, by constraining motion to avoid/ignore self-tactile information during contact. While this reduces complexity, it also limits generalization to real-world scenarios where self-contact is inevitable. Humans overcome this challenge through self-touch perception, using predictive mechanisms that anticipate the tactile consequences of their own motion, through a principle called sensory attenuation, where the nervous system differentiates predictable self-touch signals, allowing novel object stimuli to stand out as relevant. Deriving from this, we introduce TaSA, a two-phased deep predictive learning framework. In the first phase, TaSA explicitly learns self-touch dynamics, modeling how a robot's own actions generate tactile feedback. In the second phase, this learned model is incorporated into the motion learning phase, to emphasize object contact signals during manipulation. We evaluate TaSA on a set of insertion tasks, which demand fine tactile discrimination: inserting a pencil lead into a mechanical pencil, inserting coins into a slot, and fixing a paper clip onto a sheet of paper, with various orientations, positions, and sizes. Across all tasks, policies trained with TaSA achieve significantly higher success rates than baseline methods, demonstrating that structured tactile perception with self-touch based on sensory attenuation is critical for dexterous robotic manipulation.

Figures (8)

• Figure 1: Diagrammatic representation of TaSA - the proposed two-phased deep predictive learning method.
• Figure 2: Diagrammatic representation of the self-touch learning phase with input joint positions and tactile data.
• Figure 3: Comparison of the proposed method that uses both raw tactile and predicted self-touch with the baseline method that uses only raw tactile.
• Figure 4: Diagrammatic representation of successful motion of the three tasks from initiation to completion.
• Figure 5: Diagrammatic representation of the setup: We teleoperate the Allegro Hand to collect data; the clamp holds the pencil for lead insertion and the webcam records motions.