Modeling Human Strategy for Flattening Wrinkled Cloth Using Neural Networks

TL;DR

This work tackles the problem of automating cloth flattening by learning human strategies from overhead cloth images and finger actions. It collects data from human participants across four wrinkle patterns, uses image processing and PCA to extract compact wrinkle features, and trains a regression neural network to map state features to action parameters (finger position, pull length, and direction). The results show that the model can predict human actions on unseen data with low RMS errors, and reveal consistency in strategies for most wrinkle types while highlighting non-determinism in mixed wrinkles. The study demonstrates the potential of human-inspired, vision-based policies to inform robotic cloth manipulation, with implications for sewing and textiles, and sets the stage for real-time, human-guided robotic flattening.

Abstract

This paper explores a novel approach to model strategies for flattening wrinkled cloth learning from humans. A human participant study was conducted where the participants were presented with various wrinkle types and tasked with flattening the cloth using the fewest actions possible. A camera and Aruco marker were used to capture images of the cloth and finger movements, respectively. The human strategies for flattening the cloth were modeled using a supervised regression neural network, where the cloth images served as input and the human actions as output. Before training the neural network, a series of image processing techniques were applied, followed by Principal Component Analysis (PCA) to extract relevant features from each image and reduce the input dimensionality. This reduction decreased the model's complexity and computational cost. The actions predicted by the neural network closely matched the actual human actions on an independent data set, demonstrating the effectiveness of neural networks in modeling human actions for flattening wrinkled cloth.

Figures (9)

• Figure 1: Four wrinkle types used in experiments: (a) horizontal, (b) vertical, (c) inclined and (d) mixed. Black dot shows location where the cloth is pinned to the table.
• Figure 2: Experimental setup for data collection
• Figure 3: Image processing steps utilized for extracting wrinkle features.
• Figure 4: Schematic illustrating human strategy for wrinkle removal from (a) horizontal, (b) vertical, (c) inclined, and (d) mixed wrinkle types. Red dots denote the location of finger placement in the first three sub-plots where there is only one dominant region of finger placement. In the fourth sub-plot, different colors represent each of the three major regions of finger placement. Black arrows depict the length and direction of pull operation.
• Figure 5: Plot of explained variance of the input image data.