Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Granular Media Avalanche Behavior for Indirectly Manipulating Obstacles on a Granular Slope

Haodi Hu, Feifei Qian, Daniel Seita

TL;DR

This work tackles the problem of indirectly relocating obstacles on granular slopes by exploiting avalanche dynamics with legged robot actuation. It introduces GRAIN, a learning-based framework that uses a Vision Transformer to predict obstacle motions from image-based representations of the granular state and leg excavation actions, guiding a greedy manipulation policy. Real-world experiments with 100 trials show GRAIN achieving over $80\%$ success across tasks with up to four obstacles and generalizing to objects with varying physics, demonstrating the feasibility of indirect manipulation on deformable substrates. The approach advances obstacle-aided locomotion by enabling a robot to shape the substrate itself to reposition obstacles, with potential extensions to multi-step planning and more complex obstacle interactions.

Abstract

Legged robot locomotion on sand slopes is challenging due to the complex dynamics of granular media and how the lack of solid surfaces can hinder locomotion. A promising strategy, inspired by ghost crabs and other organisms in nature, is to strategically interact with rocks, debris, and other obstacles to facilitate movement. To provide legged robots with this ability, we present a novel approach that leverages avalanche dynamics to indirectly manipulate objects on a granular slope. We use a Vision Transformer (ViT) to process image representations of granular dynamics and robot excavation actions. The ViT predicts object movement, which we use to determine which leg excavation action to execute. We collect training data from 100 real physical trials and, at test time, deploy our trained model in novel settings. Experimental results suggest that our model can accurately predict object movements and achieve a success rate $\geq 80\%$ in a variety of manipulation tasks with up to four obstacles, and can also generalize to objects with different physics properties. To our knowledge, this is the first paper to leverage granular media avalanche dynamics to indirectly manipulate objects on granular slopes. Supplementary material is available at https://sites.google.com/view/grain-corl2024/home.

Learning Granular Media Avalanche Behavior for Indirectly Manipulating Obstacles on a Granular Slope

TL;DR

This work tackles the problem of indirectly relocating obstacles on granular slopes by exploiting avalanche dynamics with legged robot actuation. It introduces GRAIN, a learning-based framework that uses a Vision Transformer to predict obstacle motions from image-based representations of the granular state and leg excavation actions, guiding a greedy manipulation policy. Real-world experiments with 100 trials show GRAIN achieving over success across tasks with up to four obstacles and generalizing to objects with varying physics, demonstrating the feasibility of indirect manipulation on deformable substrates. The approach advances obstacle-aided locomotion by enabling a robot to shape the substrate itself to reposition obstacles, with potential extensions to multi-step planning and more complex obstacle interactions.

Abstract

Legged robot locomotion on sand slopes is challenging due to the complex dynamics of granular media and how the lack of solid surfaces can hinder locomotion. A promising strategy, inspired by ghost crabs and other organisms in nature, is to strategically interact with rocks, debris, and other obstacles to facilitate movement. To provide legged robots with this ability, we present a novel approach that leverages avalanche dynamics to indirectly manipulate objects on a granular slope. We use a Vision Transformer (ViT) to process image representations of granular dynamics and robot excavation actions. The ViT predicts object movement, which we use to determine which leg excavation action to execute. We collect training data from 100 real physical trials and, at test time, deploy our trained model in novel settings. Experimental results suggest that our model can accurately predict object movements and achieve a success rate in a variety of manipulation tasks with up to four obstacles, and can also generalize to objects with different physics properties. To our knowledge, this is the first paper to leverage granular media avalanche dynamics to indirectly manipulate objects on granular slopes. Supplementary material is available at https://sites.google.com/view/grain-corl2024/home.
Paper Structure (21 sections, 3 equations, 12 figures, 3 tables, 2 algorithms)

This paper contains 21 sections, 3 equations, 12 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Problem Statement
  4. Approach: GRAIN
  5. Image Representations of Granular Dynamics
  6. Training Objective for the Dynamics Model
  7. Leg Manipulation Policy
  8. Experiment Setup
  9. Experiment Results and Discussions
  10. Model Performance on Predicting Obstacle Movements
  11. Single Leg Manipulation Performance
  12. Quadruped Robot Manipulation Experiments
  13. Failure Cases and Limitations
  14. Conclusion
  15. Additional Details of GRAIN
  16. ...and 6 more sections

Figures (12)

  • Figure 1: Top: (a) our quadrupedal robot and (b) a granular slope with the robot and obstacles; (c) our setup to collect data from one robot leg (highlighted in yellow) manipulating obstacles, with our designed gantry system (highlighted in cyan) for moving the leg over the granular slope. Bottom: (d), (e), (f) an example trial of our proposed system to manipulate obstacles on the granular slope. The three blue arrows in (d) represent the change in each obstacle's location after applying the excavation, and the red shaded areas are targets.
  • Figure 2: Overview of GRAIN. (a) The ViT has three inputs: the depth image of the granular surface $\mathbf{x}_t$, the change in depth before and after excavation $\Delta \mathbf{x}_t$, and the image representation of the action $I(\mathbf{a}_t)$. The output of the ViT is the prediction of the obstacle's post-excavation location which is a 2D vector. (b) We use the trained ViT to predict object movements based on leg excavation actions. The ViT combines the current observation with candidate actions as input and predicts corresponding object movement. The ViT considers one obstacle in its output; see Sec. \ref{['ssec:leg-manip-policy']} for handling $\ge 2$ obstacles at test time. See Sec. \ref{['sec:approach']} for details on notation.
  • Figure 3: Granular flow for 2 sequential excavations. Red vectors represent the change of particle positions between two frames.
  • Figure 4: Example of $\mathbf{e}_t$ vector, pointing to the obstacle's target.
  • Figure 5: Obstacle movement with different excavation actions, visualized with colored squares (5 actions per image above). These $5\times 3 = 15$ colored squares are the 15 discretized excavation actions we consider. The corresponding solid color circles and empty color circles are the trained model's predicted obstacle positions and experiment-measured obstacle positions, respectively. (This figure is best viewed zoomed-in.)
  • ...and 7 more figures