RoPotter: Toward Robotic Pottery and Deformable Object Manipulation with Structural Priors

TL;DR

The paper tackles robotic manipulation of volumetric deformable objects (clay) under occlusion by introducing RoPotter, a pipeline that leverages structural priors to simplify learning. It develops two representations—RoPotter-2D (a 2D cross-section) and RoPotter-Mesh (ARAP-based occlusion recovery)—and trains a diffusion-based policy from demonstrations to deform clay into bowls on a pottery wheel. Across controlled experiments with varying clay masses, RoPotter-Mesh achieves significant reductions in final shape error (notably 44.4% for wide bowls) and maintains strong geometric accuracy, outperforming a 3D baseline while demonstrating robustness to occlusions. The work highlights the potential of task-specific priors to improve sample efficiency and long-horizon deformable-object manipulation, and outlines future work on goal-conditioned control and human-robot collaboration.

Abstract

Humans are capable of continuously manipulating a wide variety of deformable objects into complex shapes. This is made possible by our intuitive understanding of material properties and mechanics of the object, for reasoning about object states even when visual perception is occluded. These capabilities allow us to perform diverse tasks ranging from cooking with dough to expressing ourselves with pottery-making. However, developing robotic systems to robustly perform similar tasks remains challenging, as current methods struggle to effectively model volumetric deformable objects and reason about the complex behavior they typically exhibit. To study the robotic systems and algorithms capable of deforming volumetric objects, we introduce a novel robotics task of continuously deforming clay on a pottery wheel. We propose a pipeline for perception and pottery skill-learning, called RoPotter, wherein we demonstrate that structural priors specific to the task of pottery-making can be exploited to simplify the pottery skill-learning process. Namely, we can project the cross-section of the clay to a plane to represent the state of the clay, reducing dimensionality. We also demonstrate a mesh-based method of occluded clay state recovery, toward robotic agents capable of continuously deforming clay. Our experiments show that by using the reduced representation with structural priors based on the deformation behaviors of the clay, RoPotter can perform the long-horizon pottery task with 44.4% lower final shape error compared to the state-of-the-art baselines.

Figures (8)

• Figure 1: RoPotter Pipeline. We train a behavior cloning policy with expert demonstration data from teleoperation and point cloud state estimates of the clay. We demonstrate that the structural priors of the clay deformations and geometry can assist with the recovery of occluded points during continuous deformable object manipulation.
• Figure 2: Our setup for robotic pottery-making: two RGB-D cameras each provide a partial view of the clay, and multiple instances of the point clouds are combined to increase how much of the total surface of the clay can be observed.
• Figure 3: The pottery-making task. We demonstrate the RoPotter's ability to produce two bowls of different dimensions.
• Figure 4: Proposed pipeline for the robotic pottery. We collected expert demonstrations for the two bowl shapes using teleoperation with a gaming console controller. We paired the expert-demonstrated actions with the point cloud observation of the clay shape that was stiched together from multiple virtual perspectives.
• Figure 5: RoPotter-Mesh method outline and ablation results.