Granular Loco-Manipulation: Repositioning Rocks Through Strategic Sand Avalanche

TL;DR

DiffusiveGRAIN addresses obstacle-aided locomotion on granular slopes by jointly predicting granular avalanche dynamics and robot state changes under multi-leg excavation. It uses a diffusion-based environment predictor and a UNet-based robot state predictor, with an Effective Action Adjustment to align predictions with actual robot motion, enabling receding-horizon planning over four steps. The approach demonstrates superior loco-manipulation performance over a GRAIN baseline, achieving up to 70% success in moving closely spaced rocks to targets while maintaining locomotion goals, and shows potential for multi-robot collaboration. These results highlight a practical pathway for locomoting robots to actively shape their granular environments to improve mobility on challenging terrains, though mass effects and scalable planning remain avenues for future work.

Abstract

Legged robots have the potential to leverage obstacles to climb steep sand slopes. However, efficiently repositioning these obstacles to desired locations is challenging. Here we present DiffusiveGRAIN, a learning-based method that enables a multi-legged robot to strategically induce localized sand avalanches during locomotion and indirectly manipulate obstacles. We conducted 375 trials, systematically varying obstacle spacing, robot orientation, and leg actions in 75 of them. Results show that the movement of closely-spaced obstacles exhibits significant interference, requiring joint modeling. In addition, different multi-leg excavation actions could cause distinct robot state changes, necessitating integrated planning of manipulation and locomotion. To address these challenges, DiffusiveGRAIN includes a diffusion-based environment predictor to capture multi-obstacle movements under granular flow interferences and a robot state predictor to estimate changes in robot state from multi-leg action patterns. Deployment experiments (90 trials) demonstrate that by integrating the environment and robot state predictors, the robot can autonomously plan its movements based on loco-manipulation goals, successfully shifting closely located rocks to desired locations in over 65% of trials. Our study showcases the potential for a locomoting robot to strategically manipulate obstacles to achieve improved mobility on challenging terrains.

Figures (16)

• Figure 1: Experiment environment, with (a) a side view of the granular trackway with an inclination angle of $\Phi=20$ degrees; (b) the granular trackway with two robotic legs mounted on an actuated gantry system; (c) the robot (not the manipulator in (b)) in the granular trackway, and 3D-printed obstacles (purple semi-spheres).
• Figure 2: Left (4 images): An example of the robot flipping backward (over its two back legs) on a steep sand slope due to undesired leg-obstacle contact. The cyan rectangle highlights the leg's contact with the obstacle. Right: Stepping on undesired locations can result in a high risk of robot slipping, stuck, or flipping backwards.
• Figure 3: Experiment setup for investigating GRAIN hulearning (see Sec. \ref{['sec:challenges_grain']}). Left top: multi-obstacle manipulation experiment setup, where $\Delta dx$ is the lateral distance and $\Delta dy$ is the fore-aft distance between the two obstacles; Left bottom: the robot state change experiment, where we investigate the robot locomotion state change under 6 different combinations of leg excavation actions; Right top: 2 obstacles distance in horizontal and fore-aft directions affects obstacle movement, red dash line represents the obstacle movement without the affect of the other obstacle; Right bottom: statistics of robot state change by different robot actions.
• Figure 4: System overview. The environment predictor$f_e$ uses a diffusion model (with a U-Net backbone) to predict the depth change of the environment given the depth image and action. The robot state predictor$f_r$ uses a U-Net to predict the robot state change given the robot state and action. During policy execution, given the predicted robot state change, we introduce an "Effective Action Adjustment (EAA)" (see Sec. \ref{['ssec:EAA']}). We then combine the updated robot action image with the depth image to the trained diffusion model and get the predicted depth image change. We combine this with the predicted robot state and the original depth image to get the predicted next depth image. The red addition symbols represent channel-wise image concatenation operation, and the green addition symbols represent the image combination method as described in Sec. \ref{['ssec:policy_execution']}.
• Figure 5: An example robot loco-manipulation trial for DiffusiveGRAIN and GRAIN. The robot must bring 4 obstacles below the red horizontal line while also moving to the target marked with the green square. In DiffusiveGRAIN, the robot achieved both locomotion and manipulation at step 22. In GRAIN, the robot achieved its locomotion task but only moved the middle 2 obstacles below the red line, and thus failed in manipulation.