Learning Neural Force Manifolds for Sim2Real Robotic Symmetrical Paper Folding

TL;DR

The paper tackles robust single-manipulator folding of papers with large nonlinear deformations by coupling a physically-grounded 2D planar-rod model with scaling analysis to learn a non-dimensional neural force manifold (NFM). A neural predictor estimates the force relation and optimal grasp orientation from nondimensional coordinates, enabling global path planning via Uniform Cost Search over the NFM and real-time closed-loop model-predictive control with visual feedback. The authors demonstrate a 15× faster offline trajectory generation and significant reductions in sliding across diverse materials and geometries in extensive sim2real experiments, including extremely slick surfaces and stiff cardboard. This approach generalizes across materials and shapes, reduces reliance on hand-crafted heuristics, and lays a foundation for robust deformable-object manipulation with data-driven, physics-informed planning and real-time control.

Abstract

Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, such as imitation learning, have been used to address deformable material manipulation. These approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g., friction) properties. This article tackles a fundamental but difficult deformable manipulation task: forming a predefined fold in paper with only a single manipulator. A sim2real framework combining physically-accurate simulation and machine learning is used to train a deep neural network capable of predicting the external forces induced on the manipulated paper given a grasp position. We frame the problem using scaling analysis, resulting in a control framework robust against material and geometric changes. Path planning is then carried out over the generated ``neural force manifold'' to produce robot manipulation trajectories optimized to prevent sliding, with offline trajectory generation finishing 15$\times$ faster than previous physics-based folding methods. The inference speed of the trained model enables the incorporation of real-time visual feedback to achieve closed-loop model-predictive control. Real-world experiments demonstrate that our framework can greatly improve robotic manipulation performance compared to state-of-the-art folding strategies, even when manipulating paper objects of various materials and shapes.

Figures (15)

• Figure 1: Half valley folding for A4 paper with (a) intuitive manipulation and (b) our designed optimal manipulation. An intuitive manipulation scheme such as tracing a semicircle experiences significant sliding due to the bending stiffness of the paper, resulting in a poor fold. By contrast, our optimal manipulation approach achieves an excellent fold by taking into consideration the paper's deformation to minimize sliding.
• Figure 2: States of the paper during the folding process. The manipulation process involves two steps. The first (folding) step transitions the paper from the initial state (a), where the paper lies flat on the substrate, to the folding state (b), where the manipulated end is moved to the "crease target" line $C$. The second (creasing) step then transitions the paper from state (b) to the final folded state (c), which involves forming the desired crease on the paper.
• Figure 3: (a) Schematic of paper during the folding state. (b) Bending deformations of a small piece in the paper. (c) Reduced-order discrete model (planer rod) representation of the paper. (d) Notations in the discrete model.
• Figure 4: (a) Side view of a symmetrical paper during folding with coordinate frame and relevant notations. (b) Sampled $\lambda$ forces for a particular $\bar{l}_s$ of 4.10. This shows one of the sampled "partial" force manifolds that we use to train our neural network on.
• Figure 5: Visualization of the trained neural network's non-dimensionalized $\lambda$ force manifold $\mathcal{M}$ (a) and $\alpha$ manifold (b). An extremely low $\bar{\delta}$ discretization is used to showcase smoothness. For the force manifold, we observe two distinctive local minima canyons. Note that regions outside the workspace $\mathcal{W}$ are physically inaccurate, but are of no consequence as they are ignored. For the $\alpha$ manifold, we observe continuous smooth interpolation throughout, which is crucial for producing feasible trajectories. Both manifolds showcase the trajectories used in the experiments for folding paper in half for $L_{gb} \in [0.048, 0.060, 0.132]$. (c) The three trajectories in (a) and (b) scaled back to real space. These are the actual trajectories used by the robot. (d) Arbitrary trajectories for various $L_{gb}$ with identical start and goal states, highlighting the effect of the material property on our control policy.