DOFS: A Real-world 3D Deformable Object Dataset with Full Spatial Information for Dynamics Model Learning

TL;DR

This work proposes DOFS, a pilot dataset of 3D deformable objects (DOs) with full spatial information with full spatial information (i.e., top, side, and bottom information) using a novel and low-cost data collection platform with a transparent operating plane.

Abstract

This work proposes DOFS, a pilot dataset of 3D deformable objects (DOs) (e.g., elasto-plastic objects) with full spatial information (i.e., top, side, and bottom information) using a novel and low-cost data collection platform with a transparent operating plane. The dataset consists of active manipulation action, multi-view RGB-D images, well-registered point clouds, 3D deformed mesh, and 3D occupancy with semantics, using a pinching strategy with a two-parallel-finger gripper. In addition, we trained a neural network with the down-sampled 3D occupancy and action as input to model the dynamics of an elasto-plastic object. Our dataset and all CADs of the data collection system will be released soon on our website.

Figures (6)

• Figure 1: Dynamics difference caused by the hollow bottom. For two pieces of plasticine of the same appearance and size, one is solid and another is hollow, the deformation results are completely different with the same action.
• Figure 2: Hardware setup of our data collection platform. The blue part is a transparent acrylic board that serves as an operating plane. The orange parts are 4 cameras above the plane to collect top and side information. The green parts are 2 cameras below the plane to collect bottom-side information.
• Figure 3: Six RGB-D images captured from RealSense D435i. Four images from the cameras installed above the operating plane show the top and side information of the plasticine. Two images from the cameras installed below the operating plane show the bottom side information of the plasticine.
• Figure 4: Data visualization of one frame during manipulation. (a): Well-registered point cloud of pinched plasticine without background. (b): Reconstructed deformed mesh. (c): 3D occupancy of plasticine.
• Figure 5: Visualization of Full spatial information collection. (a): We embed the cartoon model into the bottom of the plasticine and take it out. (b): The visualization of the point cloud of plasticine. (c): The deformed mesh of plasticine.