Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Behavioral Learning of Dish Rinsing and Scrubbing based on Interruptive Direct Teaching Considering Assistance Rate

Shumpei Wakabayashi, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

TL;DR

This work tackles safe, dexterous robotic dishwashing by coupling interruptive direct teaching with an autoregressive, LSTM-based dynamics model that includes a human-assistance variable $p$. The model is trained on trajectories augmented by human corrections, and trajectory inputs are optimized via backpropagation to minimize reliance on assistance, encouraging actions that avoid splashing, dish damage, and target object motion. Path-planning validation and real dishwashing experiments demonstrate that scrubbing and rinsing can adapt to unknown dishware while reducing human intervention, achieving moderate scrubbing force and thorough rinsing. Overall, the approach offers a task-focused, data-efficient route to autonomous, safe manipulation in unstructured object scenarios.

Abstract

Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of the object by estimating the state of the object and the robot itself, the control input, and the amount of human assistance required (assistance rate) after the human corrects the initial trajectory of the robot's hands by interruptive direct teaching. By backpropagating the error between the estimated and the reference value using the acquired dynamics model, the robot can generate a control input that approaches the reference value, for example, so that human assistance is not required and the dish does not move excessively. This allows for adaptive rinsing and scrubbing of dishes with unknown shapes and properties. As a result, it is possible to generate safe actions that require less human assistance.

Behavioral Learning of Dish Rinsing and Scrubbing based on Interruptive Direct Teaching Considering Assistance Rate

TL;DR

This work tackles safe, dexterous robotic dishwashing by coupling interruptive direct teaching with an autoregressive, LSTM-based dynamics model that includes a human-assistance variable . The model is trained on trajectories augmented by human corrections, and trajectory inputs are optimized via backpropagation to minimize reliance on assistance, encouraging actions that avoid splashing, dish damage, and target object motion. Path-planning validation and real dishwashing experiments demonstrate that scrubbing and rinsing can adapt to unknown dishware while reducing human intervention, achieving moderate scrubbing force and thorough rinsing. Overall, the approach offers a task-focused, data-efficient route to autonomous, safe manipulation in unstructured object scenarios.

Abstract

Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of the object by estimating the state of the object and the robot itself, the control input, and the amount of human assistance required (assistance rate) after the human corrects the initial trajectory of the robot's hands by interruptive direct teaching. By backpropagating the error between the estimated and the reference value using the acquired dynamics model, the robot can generate a control input that approaches the reference value, for example, so that human assistance is not required and the dish does not move excessively. This allows for adaptive rinsing and scrubbing of dishes with unknown shapes and properties. As a result, it is possible to generate safe actions that require less human assistance.
Paper Structure (21 sections, 6 equations, 19 figures, 2 tables, 2 algorithms)

This paper contains 21 sections, 6 equations, 19 figures, 2 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. AUTOREGRESSIVE MODEL LEARNING WITH INTERRUPTIVE HUMAN ASSISTANCE AND MANIPULATION MODIFICATION DURING OPTIMIZATION
  3. Overview of an Autoregressive Model and Optimization of Control Inputs
  4. Training an Autoregressive Model
  5. Trajectory Optimization by Back Propagation of an Autoregressive Model
  6. Detailis of an Autoregressive Model
  7. PATH PLANNING FOR AUTOREGRESSIVE MODEL VALIDATION
  8. Dataset Collection and Model Training
  9. Optimization and Results for Agent Behavior
  10. DISH WASHING EXPERIMENTS
  11. Collection of Datasets for Water Rinsing and Scrubbing of Tableware
  12. Initial Trajectory Generation Method
  13. Manipulation correction with temporary human assistance
  14. Data Collection and Analysis
  15. Training of the Autoregressive Model for Water Rinsing and Scrubbing of Tableware
  16. ...and 6 more sections

Figures (19)

  • Figure 1: In order to manipulate an object safe and dexterous way, the robot executes operations based on a geometric state of the object, but sometimes modified by human assistance. With the robot states and object states, the control input, and the amount of human assistance, the robot acquires how to manipulate it. After training, the robot can automatically scrub with adjusting force and rinse object avoiding splashing water.
  • Figure 2: Training phase: Collecting the datasets consisted of robot states, object states and control input. Calculating the loss with predicted data and the actual data.
  • Figure 3: Optimization phase: Modifying the angle vectors of the arms with reference using backpropagation of the network.
  • Figure 4: Path planning for the autoregressive model validation. A red agent aims to reach goal from top left to bottom right avoiding a dynamic green obstacle. The pictures show the moment when the agent reaches goal. When both $u$ and $p$ are backpropageted, the agents avoid obstacle in a small circle and reaches goal.
  • Figure 5: The left picture is the dishes for washing training. The dishes are including various size and shape of plate, fork and spoon. The right picture is the dirty dishes for test. They are different from the dishes for training, but their domain are similar.
  • ...and 14 more figures