Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation

TL;DR

Rock capturing with standard excavator buckets is highly challenging due to complex interactions with granular material. The authors train a model-free PPO policy in a high-fidelity AGX Dynamics simulation, with extensive domain randomization over rock geometry, density, and initial configurations, to directly output joint speeds for the boom, arm, and bucket. The resulting controller demonstrates robust generalization to unseen rocks and materials, achieving a high success rate around $0.8$ and performing comparably to human participants in several scenarios without specialized hardware or explicit material models. This work establishes the feasibility of learning-based excavation strategies for discrete object manipulation and points to practical automation potential in construction and mining, while outlining steps to bridge the sim-to-real gap and enhance safety.

Abstract

Rock capturing with standard excavator buckets is a challenging task typically requiring the expertise of skilled operators. Unlike soil digging, it involves manipulating large, irregular rocks in unstructured environments where complex contact interactions with granular material make model-based control impractical. Existing autonomous excavation methods focus mainly on continuous media or rely on specialized grippers, limiting their applicability to real-world construction sites. This paper introduces a fully data-driven control framework for rock capturing that eliminates the need for explicit modeling of rock or soil properties. A model-free reinforcement learning agent is trained in the AGX Dynamics simulator using the Proximal Policy Optimization (PPO) algorithm and a guiding reward formulation. The learned policy outputs joint velocity commands directly to the boom, arm, and bucket of a CAT365 excavator model. Robustness is enhanced through extensive domain randomization of rock geometry, density, and mass, as well as the initial configurations of the bucket, rock, and goal position. To the best of our knowledge, this is the first study to develop and evaluate an RL-based controller for the rock capturing task. Experimental results show that the policy generalizes well to unseen rocks and varying soil conditions, achieving high success rates comparable to those of human participants while maintaining machine stability. These findings demonstrate the feasibility of learning-based excavation strategies for discrete object manipulation without requiring specialized hardware or detailed material models.

Figures (21)

• Figure 1: Rock capturing task using an excavator Liangjun2021autonomous.
• Figure 2: The pipeline for training of the control policy.
• Figure 3: Illustration of the maximum and minimum reach capabilities of the manipulator at ground level. The red boundary indicates the excavator's effective workspace where the rock capturing task should be performed.
• Figure 4: Illustration of randomly sampled goal positions during training. The samples are drawn from the bivariate normal distribution $\mathcal{N} \left(\boldsymbol{\mu},\boldsymbol{\Sigma}\right)$ and constrained within a circular region of radius $0.3~m$.
• Figure 5: The geometries of the rocks used during training. Each episode randomly selects one of the two rock meshes.