Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Supervised Learning of Visual Servoing for Low-Rigidity Robots Considering Temporal Body Changes

Kento Kawaharazuka, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

TL;DR

This work tackles visual servoing for low-rigidity robots whose bodies change over time, causing calibration drift. It introduces Visual Servoing Network with Parametric Bias (VSNPB), a self-supervised framework that learns body-aware servoing and embeds temporal dynamics into a 2D parametric bias, updated online to track current body state. Autonomous data collection leverages the reproducibility of the robot's motions, while online PB updates allow adaptation to unseen body states, enabling reliable grasping across objects. The approach demonstrates high success when the PB matches the current body state and remains functional through online adaptation, highlighting practical potential for autonomously evolving, flexible robots in real-world tasks.

Abstract

In this study, we investigate object grasping by visual servoing in a low-rigidity robot. It is difficult for a low-rigidity robot to handle its own body as intended compared to a rigid robot, and calibration between vision and body takes some time. In addition, the robot must constantly adapt to changes in its body, such as the change in camera position and change in joints due to aging. Therefore, we develop a method for a low-rigidity robot to autonomously learn visual servoing of its body. We also develop a mechanism that can adaptively change its visual servoing according to temporal body changes. We apply our method to a low-rigidity 6-axis arm, MyCobot, and confirm its effectiveness by conducting object grasping experiments based on visual servoing.

Self-Supervised Learning of Visual Servoing for Low-Rigidity Robots Considering Temporal Body Changes

TL;DR

This work tackles visual servoing for low-rigidity robots whose bodies change over time, causing calibration drift. It introduces Visual Servoing Network with Parametric Bias (VSNPB), a self-supervised framework that learns body-aware servoing and embeds temporal dynamics into a 2D parametric bias, updated online to track current body state. Autonomous data collection leverages the reproducibility of the robot's motions, while online PB updates allow adaptation to unseen body states, enabling reliable grasping across objects. The approach demonstrates high success when the PB matches the current body state and remains functional through online adaptation, highlighting practical potential for autonomously evolving, flexible robots in real-world tasks.

Abstract

In this study, we investigate object grasping by visual servoing in a low-rigidity robot. It is difficult for a low-rigidity robot to handle its own body as intended compared to a rigid robot, and calibration between vision and body takes some time. In addition, the robot must constantly adapt to changes in its body, such as the change in camera position and change in joints due to aging. Therefore, we develop a method for a low-rigidity robot to autonomously learn visual servoing of its body. We also develop a mechanism that can adaptively change its visual servoing according to temporal body changes. We apply our method to a low-rigidity 6-axis arm, MyCobot, and confirm its effectiveness by conducting object grasping experiments based on visual servoing.
Paper Structure (13 sections, 1 equation, 9 figures)

This paper contains 13 sections, 1 equation, 9 figures.

Table of Contents

  1. INTRODUCTION
  2. Self-Supervised Learning of Visual Servoing Considering Temporal Body Changes
  3. Network Structure of VSNPB
  4. Data Collection
  5. Training of VSNPB
  6. Online Update of Parametric Bias
  7. Visual Servoing Using VSNPB
  8. Experiments
  9. Experimental Setup
  10. Data Collection and Training Experiment
  11. Visual Servoing Experiment
  12. Discussion
  13. CONCLUSION

Figures (9)

  • Figure 1: The concept of this study: self-supervised learning of visual servoing and handling of temporal body changes for low-rigidity robots.
  • Figure 2: The overall system of visual servoing network with parametric bias.
  • Figure 3: The experimental setup of the low-rigidity robot MyCobot and four grasped objects.
  • Figure 4: The temporal body changes handled in this study: change in realization of joint angle and change in camera position.
  • Figure 5: The procedure of data collection for visual servoing.
  • ...and 4 more figures