Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Visuo-Tactile Exploration of Unknown Rigid 3D Curvatures by Vision-Augmented Unified Force-Impedance Control

Kübra Karacan, Anran Zhang, Hamid Sadeghian, Fan Wu, Sami Haddadin

TL;DR

The paper tackles the challenge of deploying torque-controlled tactile robots in manufacturing by enabling intuitive visuo-tactile exploration of unknown 3D curvatures. It introduces VA-UFIC, a framework that fuses vision and touch to adapt stiffness and force in real time, underpinned by online contact alignment monitoring and virtual energy tanks that guarantee passivity. Key contributions include a solid control design blending impedance and force channels, a robust visuo-tactile alignment mechanism using depth data and PCA-based surface normals, and a stability analysis with energy-tank augmentations across contact states, all validated on a Franka Emika platform. The results show accurate contact alignment, manageable latency, and stable force regulation during curvature exploration, with limitations around highly irregular geometries and camera field-of-view; future work aims to plan object-centric tactile policies to broaden manufacturing applicability.

Abstract

Despite recent advancements in torque-controlled tactile robots, integrating them into manufacturing settings remains challenging, particularly in complex environments. Simplifying robotic skill programming for non-experts is crucial for increasing robot deployment in manufacturing. This work proposes an innovative approach, Vision-Augmented Unified Force-Impedance Control (VA-UFIC), aimed at intuitive visuo-tactile exploration of unknown 3D curvatures. VA-UFIC stands out by seamlessly integrating vision and tactile data, enabling the exploration of diverse contact shapes in three dimensions, including point contacts, flat contacts with concave and convex curvatures, and scenarios involving contact loss. A pivotal component of our method is a robust online contact alignment monitoring system that considers tactile error, local surface curvature, and orientation, facilitating adaptive adjustments of robot stiffness and force regulation during exploration. We introduce virtual energy tanks within the control framework to ensure safety and stability, effectively addressing inherent safety concerns in visuo-tactile exploration. Evaluation using a Franka Emika research robot demonstrates the efficacy of VA-UFIC in exploring unknown 3D curvatures while adhering to arbitrarily defined force-motion policies. By seamlessly integrating vision and tactile sensing, VA-UFIC offers a promising avenue for intuitive exploration of complex environments, with potential applications spanning manufacturing, inspection, and beyond.

Visuo-Tactile Exploration of Unknown Rigid 3D Curvatures by Vision-Augmented Unified Force-Impedance Control

TL;DR

The paper tackles the challenge of deploying torque-controlled tactile robots in manufacturing by enabling intuitive visuo-tactile exploration of unknown 3D curvatures. It introduces VA-UFIC, a framework that fuses vision and touch to adapt stiffness and force in real time, underpinned by online contact alignment monitoring and virtual energy tanks that guarantee passivity. Key contributions include a solid control design blending impedance and force channels, a robust visuo-tactile alignment mechanism using depth data and PCA-based surface normals, and a stability analysis with energy-tank augmentations across contact states, all validated on a Franka Emika platform. The results show accurate contact alignment, manageable latency, and stable force regulation during curvature exploration, with limitations around highly irregular geometries and camera field-of-view; future work aims to plan object-centric tactile policies to broaden manufacturing applicability.

Abstract

Despite recent advancements in torque-controlled tactile robots, integrating them into manufacturing settings remains challenging, particularly in complex environments. Simplifying robotic skill programming for non-experts is crucial for increasing robot deployment in manufacturing. This work proposes an innovative approach, Vision-Augmented Unified Force-Impedance Control (VA-UFIC), aimed at intuitive visuo-tactile exploration of unknown 3D curvatures. VA-UFIC stands out by seamlessly integrating vision and tactile data, enabling the exploration of diverse contact shapes in three dimensions, including point contacts, flat contacts with concave and convex curvatures, and scenarios involving contact loss. A pivotal component of our method is a robust online contact alignment monitoring system that considers tactile error, local surface curvature, and orientation, facilitating adaptive adjustments of robot stiffness and force regulation during exploration. We introduce virtual energy tanks within the control framework to ensure safety and stability, effectively addressing inherent safety concerns in visuo-tactile exploration. Evaluation using a Franka Emika research robot demonstrates the efficacy of VA-UFIC in exploring unknown 3D curvatures while adhering to arbitrarily defined force-motion policies. By seamlessly integrating vision and tactile sensing, VA-UFIC offers a promising avenue for intuitive exploration of complex environments, with potential applications spanning manufacturing, inspection, and beyond.
Paper Structure (12 sections, 37 equations, 3 figures, 1 table)

This paper contains 12 sections, 37 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Control Design
  4. Visuo-Tactile Exploration of Unknown Rigid 3D Curvatures by VA-UFIC
  5. Passivity-Based Stability Analysis and Installing Virtual Energy Tanks
  6. Case I: Stability analysis without any contact
  7. Case II: Stability analysis during contact
  8. Experimental Validation
  9. Experimental Procedure
  10. Performance Metrics
  11. Results and Discussion
  12. Conclusion

Figures (3)

  • Figure 1: Visuo-Tactile Exploration of Unknown Rigid 3D Curvatures by vision-augmented unified force-impedance control (VA-UFIC) for a chosen tactile skill. Visuo-tactile exploration is the next step to achieving a force-motion planning framework that outputs an object-centric force-motion profile for an arbitrary tactile skill policy. The explored environment is fed back to the library to further plan the force-motion policy.
  • Figure 2: Visuo-Tactile Exploration of an Unknown Rigid 3D Curvature. The robot's actual trajectory along the y- and z-direction in the base frame is compared to the model of the contact surface.
  • Figure 3: Performance Metrics Results for Visuo-Tactile Exploration during Wiping. a) Controller shaping functions, b) Desired vs. actual motion in the base frame, c) Desired force of 15N shaped by the function $\rho_\mathrm{frc}$ vs. measured force in the end effector frame, d) Tank energies.