Autonomous Quilt Spreading for Caregiving Robots
Yuchun Guo, Zhiqing Lu, Yanling Zhou, Xin Jiang
TL;DR
This work tackles autonomous caregiving by enabling a robot to re-cover an infant with a quilt despite limb-induced interference. It combines skeletal detection (DWPose), instance segmentation (Segment Anything), and a learned fabric-dynamics model (EM*D) to recognize fabric states, resolve interference, and plan spreading actions; data are generated in Blender and validated through simulations and real-world tests. Key innovations include a depth-based loss for EM*D, GA-based planning to avoid local minima, and a pruning strategy that targets low-coverage regions. The approach demonstrates reliable re-covering under interference using RGB images, with future work aiming to incorporate thermal imaging to improve infant-quilt segmentation accuracy.
Abstract
In this work, we propose a novel strategy to ensure infants, who inadvertently displace their quilts during sleep, are promptly and accurately re-covered. Our approach is formulated into two subsequent steps: interference resolution and quilt spreading. By leveraging the DWPose human skeletal detection and the Segment Anything instance segmentation models, the proposed method can accurately recognize the states of the infant and the quilt over her, which involves addressing the interferences resulted from an infant's limbs laid on part of the quilt. Building upon prior research, the EM*D deep learning model is employed to forecast quilt state transitions before and after quilt spreading actions. To improve the sensitivity of the network in distinguishing state variation of the handled quilt, we introduce an enhanced loss function that translates the voxelized quilt state into a more representative one. Both simulation and real-world experiments validate the efficacy of our method, in spreading and recover a quilt over an infant.