Disentangling perception and reasoning for improving data efficiency in learning cloth manipulation without demonstrations

TL;DR

This work tackles cloth manipulation by addressing data inefficiency in reinforcement learning through disentangling perception and reasoning. It trains a state-based, compact agent in full-state cloth simulation using offline pre-training, multi-objective objectives, and a Q-learning framework, then transfers knowledge to the real world via cross-modality distillation to a vision-based policy. Key contributions include substantial performance gains with a smaller network, detailed ablation studies of design choices, and a practical sim-to-real transfer pipeline that reduces training time to roughly 40 hours. The approach demonstrates that perception is a major bottleneck in cloth manipulation and provides a modular, scalable path toward real-world deployment of deformable-object manipulators.

Abstract

Cloth manipulation is a ubiquitous task in everyday life, but it remains an open challenge for robotics. The difficulties in developing cloth manipulation policies are attributed to the high-dimensional state space, complex dynamics, and high propensity to self-occlusion exhibited by fabrics. As analytical methods have not been able to provide robust and general manipulation policies, reinforcement learning (RL) is considered a promising approach to these problems. However, to address the large state space and complex dynamics, data-based methods usually rely on large models and long training times. The resulting computational cost significantly hampers the development and adoption of these methods. Additionally, due to the challenge of robust state estimation, garment manipulation policies often adopt an end-to-end learning approach with workspace images as input. While this approach enables a conceptually straightforward sim-to-real transfer via real-world fine-tuning, it also incurs a significant computational cost by training agents on a highly lossy representation of the environment state. This paper questions this common design choice by exploring an efficient and modular approach to RL for cloth manipulation. We show that, through careful design choices, model size and training time can be significantly reduced when learning in simulation. Furthermore, we demonstrate how the resulting simulation-trained model can be transferred to the real world. We evaluate our approach on the SoftGym benchmark and achieve significant performance improvements over available baselines on our task, while using a substantially smaller model.

Figures (4)

• Figure 1: Overview of our method. We train an optimal agent in simulation with a maximal access to the cloth's state. When transfering our agent to the real world, we distill the knowledge from the state-based simulation agent to a vision-based real world agent through the ad-hoc generation of a dataset, densely labelled by the simulation agent
• Figure 2: Top row: Random states of the cloth. Bottom row: Corresponding state images. The color of each pixel in the state image is computed from the Euclidean position of the corresponding particle in the cloth's particle grid. In order to maximise saturation in this figure, the state images presented are normalized along each channel.
• Figure 3: (a) Real-world setup for the cloth manipulation task. (b) Successful examples of real-world roll-outs with corresponding normalised improvement
• Figure 4: Qualitative evaluations of the training parameters. (a) Sample efficiency plot of pixel-space picking and node-space picking. (b) Scaling law of data for offline pre-training. (c) Impact on multi-objective training on final performance. For (b) and (c) the mean performance value is represented with its bootstrapping confidence interval