DRAPER: Towards a Robust Robot Deployment and Reliable Evaluation for Quasi-Static Pick-and-Place Cloth-Shaping Neural Controllers

TL;DR

This work tackles the challenge of reliably evaluating and deploying cloth-manipulation policies learned in simulation to real robots. It introduces the DRAPER deployment framework, which integrates real-world grasping uncertainty into simulation, standardizes vision processing, and employs a tweezers-extended gripper to improve reliability. It also adapts a diffusion-policy for quasi-static pick-and-place in towel flattening and folding, demonstrating robust real-world and simulated performance across multiple robots and fabrics. Through extensive real-world experiments, the paper provides the first comprehensive cross-method, cross-platform comparison and offers practical guidance for reliable cloth manipulation in real-world settings.

Abstract

Comparing robotic cloth-manipulation systems in a real-world setup is challenging. The fidelity gap between simulation-trained cloth neural controllers and real-world operation hinders the reliable deployment of these methods in physical trials. Inconsistent experimental setups and hardware limitations among different approaches obstruct objective evaluations. This study demonstrates a reliable real-world comparison of different simulation-trained neural controllers on both flattening and folding tasks with different types of fabrics varying in material, size, and colour. We introduce the DRAPER framework to enable this comprehensive study, which reliably reflects the true capabilities of these neural controllers. It specifically addresses real-world grasping errors, such as misgrasping and multilayer grasping, through real-world adaptations of the simulation environment to provide data trajectories that closely reflect real-world grasping scenarios. It also employs a special set of vision processing techniques to close the simulation-to-reality gap in the perception. Furthermore, it achieves robust grasping by adopting a tweezer-extended gripper and a grasping procedure. We demonstrate DRAPER's generalisability across different deep-learning methods and robotic platforms, offering valuable insights to the cloth manipulation research community.

Figures (3)

• Figure 1: Robust Folding and Flattening Deployment of Neural Controllers on Various Robot Platforms. We examine the performance of these controllers on 6 different towels, varying in terms of material, colour and length-to-width ratios.
• Figure 2: DRAPER framework that deploys RGB-D Diffusion Policy to Real-Robot Setups. Our real-world adaptation of Oracle Towel Smoothing (RealAdapt-OTS) policy is shown as green arrows on the trajectory generated by the proposed RealAdapt Towels simulation environment in the yellow box. The first and third row trajectories from the top show the vision input and actions (blue arrows) induced by the diffusion policy. In the top row, diffusion-generated pick positions are marked by green dots and place positions by red crosses with lighter colours indicating earlier denoising steps. Purple dots in the "Graspin" images at the second row represent readjusted pick positions on eroded mask images, and the red line indicates gripper orientation. The neural controller preprocesses the RGB (3rd row) and depth images (1st row) with the JA-TN vision processing strategy (red box) kadi2024mjtn. Performance is evaluated using Normalised Coverage (NC) for flattening and Intersection-over-Union (IoU) for folding. The last two trajectories demonstrate synchronised pick-and-place manipulation by UR3e and Panda robot arms for flattening and folding individually. Neural controllers can learn from the offline data collected by RealAdapt-OTS policy in our improved RealAdapt Towels simulated benchmark environment for robust deployment to reality using our grasping protocol.
• Figure 3: DRAPER Ablation Study in Simulation Environments with Various Controllers. A trajectory is regarded as a failure if a policy cannot flatten the fabrics and achieve a normalised coverage (NC) above 99% within 30 steps. By default, all neural controllers are trained using DRAPER's training strategy. Note that each colour bar represents a single trained policy examined in these three simulation environments. The newly proposed flattening Oracle policy RealAdapt-OTS consistently outperforms the old Oracle Towel Smoothing (OTS). Neural controllers trained with RealAdapt-OTS in RealAdapt Towels generalise better than those trained with OTS in Rainbow Rectangular Fabrics(OTS in RRN). Although naive depth processing (Naive Depth) can help the neural controller improve its performance on the benchmark environment it trains on, it generalises poorly compared to the one trained with DRAPER's setting. Among the neural controllers, JA-TN shows the best generalisability across the tested simulation benchmarks. Depth-Only diffusion policy and JA-TN demonstrate the best generalisability among the neural controllers.