Leveraging Simulation-Based Model Preconditions for Fast Action Parameter Optimization with Multiple Models

TL;DR

The paper tackles fast and accurate robotic action-parameter optimization for manipulation tasks that involve varying object geometries and materials. It proposes a multi-model framework that selects among analytical, learned, and simulation predictors using Model Deviation Estimators (MDEs) to identify model preconditions and to pick the fastest reliable model during each optimization step, formalized within a one-step GOAL-based reinforcement learning setting. A key contribution is the introduction of sim-to-sim (S2S) MDEs, along with vision-based inputs using heightmaps and material-property masks to enable data-efficient sim-to-real (S2R) transfer via fine-tuning; S2S MDEs are trained on simulation data and can be quickly adapted to real-world data. Experimental results on a plating/food-arrangement task show that S2S MDEs can accelerate parameter optimization (roughly 15 seconds versus over 240 seconds for purely high-fidelity simulation) while maintaining high predictive accuracy, and that fine-tuning S2S MDEs to S2R MDEs enables effective real-world deployment with limited data.

Abstract

Optimizing robotic action parameters is a significant challenge for manipulation tasks that demand high levels of precision and generalization. Using a model-based approach, the robot must quickly reason about the outcomes of different actions using a predictive model to find a set of parameters that will have the desired effect. The model may need to capture the behaviors of rigid and deformable objects, as well as objects of various shapes and sizes. Predictive models often need to trade-off speed for prediction accuracy and generalization. This paper proposes a framework that leverages the strengths of multiple predictive models, including analytical, learned, and simulation-based models, to enhance the efficiency and accuracy of action parameter optimization. Our approach uses Model Deviation Estimators (MDEs) to determine the most suitable predictive model for any given state-action parameters, allowing the robot to select models to make fast and precise predictions. We extend the MDE framework by not only learning sim-to-real MDEs, but also sim-to-sim MDEs. Our experiments show that these sim-to-sim MDEs provide significantly faster parameter optimization as well as a basis for efficiently learning sim-to-real MDEs through finetuning. The ease of collecting sim-to-sim training data also allows the robot to learn MDEs based directly on visual inputs and local material properties.

Figures (12)

• Figure 1: (Top) Illustration of the plating action. The robot is equipped with a wrist camera and a vacuum gripper. (Bottom) The robot is given initial and target scenes; the task is to find the best action to reach the target scene.
• Figure 2: Overview of our framework. Given an initial state, the robot employs an optimizer guided by a reward function to predict the optimal action to achieve a target state. Throughout this optimization process, the robot leverages MDEs to dynamically select the most appropriate model from a family of predictive models.
• Figure 3: (Left) Neural network architecture of the learned model. (Right) Neural network architecture of the MDE models.
• Figure 4: Isaac Gym Simulation model. The plate and fries are initialized at the start of the simulation. The steak is placed at the target location, marked by the red X, using a tray that only contacts the steak.
• Figure 5: (Left) Four steaks are cooked rare to allow deformable behavior. Two T-bone steaks are cooked well-done to form rigid behavior. (Middle) Deformable and rigid steak comparison on the plate and on the robot vacuum gripper. (Right) Example real-world initial and target scenes.