Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiview Progress Prediction of Robot Activities

Elena Zoppellari, Federico Becattini, Marco Fiorucci, Lamberto Ballan

TL;DR

This paper proposes a multi-view architecture for action progress prediction in robot manipulation tasks and demonstrates the effectiveness of the proposed approach on Mobile ALOHA.

Abstract

For robots to operate effectively and safely alongside humans, they must be able to understand the progress of ongoing actions. This ability, known as action progress prediction, is critical for tasks ranging from timely assistance to autonomous decision-making. However, modeling action progression in robotics has often been overlooked. Moreover, a single camera may be insufficient for understanding robot's ego-actions, as self-occlusion can significantly hinder perception and model performance. In this paper, we propose a multi-view architecture for action progress prediction in robot manipulation tasks. Experiments on Mobile ALOHA demonstrate the effectiveness of the proposed approach.

Multiview Progress Prediction of Robot Activities

TL;DR

This paper proposes a multi-view architecture for action progress prediction in robot manipulation tasks and demonstrates the effectiveness of the proposed approach on Mobile ALOHA.

Abstract

For robots to operate effectively and safely alongside humans, they must be able to understand the progress of ongoing actions. This ability, known as action progress prediction, is critical for tasks ranging from timely assistance to autonomous decision-making. However, modeling action progression in robotics has often been overlooked. Moreover, a single camera may be insufficient for understanding robot's ego-actions, as self-occlusion can significantly hinder perception and model performance. In this paper, we propose a multi-view architecture for action progress prediction in robot manipulation tasks. Experiments on Mobile ALOHA demonstrate the effectiveness of the proposed approach.
Paper Structure (5 sections, 3 figures, 4 tables)

This paper contains 5 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Experiments
  5. Conclusions

Figures (3)

  • Figure 1: Proposed architecture
  • Figure 2: Progress estimated from different cameras: left (green), central (red) and right (cyan). A perfect progress estimation would exhibit a straight diagonal line from bottom left to top right.
  • Figure 3: Progress estimated with multiple cameras. Two versions are shown: trained on full videos (red) and trained on semgments (green). A perfect progress estimation would exhibit a straight diagonal line from bottom left to top right.