Prescribed Performance Control of Deformable Object Manipulation in Spatial Latent Space

TL;DR

This work tackles deformable object manipulation with an effectively infinite-dimensional state space by introducing a model-free control framework that represents deformable shapes through spatial key points extracted from point clouds, forming a spatial latent space with $\mathbf p \in \mathbb{R}^{3n}$ embedded in $\mathbb{R}^{3N}$.A Jacobian-based prescribed performance controller is developed, combining Barrier Lyapunov functions and a transformed-error scheme to bound key-point errors within predefined boundaries, while an adaptive radial-basis-function neural network estimates the unknown deformation Jacobian $\mathbf J$.Theoretical guarantees are provided via Lyapunov analysis, showing semiglobal uniform ultimate boundedness of the closed-loop system, ensuring stability and constraint satisfaction; empirical results on UR3 robots validate improved transient and steady-state performance, robustness to deformation, and spatial-information preservation in the latent representation.Limitations include occlusion sensitivity, with future work proposed to address this using temporal networks and generative adversarial networks to maintain performance under partial observations.

Abstract

Manipulating three-dimensional (3D) deformable objects presents significant challenges for robotic systems due to their infinite-dimensional state space and complex deformable dynamics. This paper proposes a novel model-free approach for shape control with constraints imposed on key points. Unlike existing methods that rely on feature dimensionality reduction, the proposed controller leverages the coordinates of key points as the feature vector, which are extracted from the deformable object's point cloud using deep learning methods. This approach not only reduces the dimensionality of the feature space but also retains the spatial information of the object. By extracting key points, the manipulation of deformable objects is simplified into a visual servoing problem, where the shape dynamics are described using a deformation Jacobian matrix. To enhance control accuracy, a prescribed performance control method is developed by integrating barrier Lyapunov functions (BLF) to enforce constraints on the key points. The stability of the closed-loop system is rigorously analyzed and verified using the Lyapunov method. Experimental results further demonstrate the effectiveness and robustness of the proposed method.

Figures (10)

• Figure 1: The configuration of 3D deformable object manipulation with constraints of key points.
• Figure 2: Structure of our proposed method.
• Figure 3: Structure of Key-Grid. In the encoder section, the key points are extracted from the input point cloud by utilizing the PointNet++. The detected key points are then used to form grid heatmaps. In the decoder section, each layer of the PointNet++, heatmaps, and MLPs are utilized to reconstruct the input. The farthest point loss of the key points and the reconstructed point cloud similarity loss are used for network training.
• Figure 4: Experiment setup and the experimental objects. (a) the eye-to-hand robot-camera platform; (b) the object manipulated in task A (A piece of sponge with a slit incision in the middle, allowing it to be stretched) with the corresponding key points; (c) the object manipulated in task B (A sponge with a square incision) with the corresponding key points; (d) the object manipulated in task C (An "L"-shape sponge) with the corresponding key points.
• Figure 5: Comparison of key point extraction. Compared with the other three learning-based methods, the key points extracted by Key-Grid have a more uniform distribution on the point cloud, and the key points have better spatiotemporal consistency. For the learning-free method FPFH, although the key points can be evenly distributed on the surface of the object, the corresponding spatial relationship is not preserved at all.

Theorems & Definitions (3)

• Lemma 1
• Theorem 1
• proof