Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging the Sim-to-Real Gap with Bayesian Inference

Jonas Rothfuss, Bhavya Sukhija, Lenart Treven, Florian Dörfler, Stelian Coros, Andreas Krause

TL;DR

It is empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification in Sim-FSVGD, and demonstrates the effectiveness of Sim-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.

Abstract

We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, SIM-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, SIM-FSVGD scales and excels also when more data is available. We empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system. Using model-based RL, we demonstrate a highly dynamic parking maneuver with drifting, using less than half the data compared to the state of the art.

Bridging the Sim-to-Real Gap with Bayesian Inference

TL;DR

It is empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification in Sim-FSVGD, and demonstrates the effectiveness of Sim-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.

Abstract

We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, SIM-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, SIM-FSVGD scales and excels also when more data is available. We empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system. Using model-based RL, we demonstrate a highly dynamic parking maneuver with drifting, using less than half the data compared to the state of the art.
Paper Structure (21 sections, 5 equations, 7 figures, 2 algorithms)

This paper contains 21 sections, 5 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work.
  3. Bridging the sim-to-real gap
  4. Neural Networks for System Identification
  5. Bayesian Neural Networks
  6. Background
  7. Learning Dynamics Models with BNNs
  8. BNN Inference in the Function Space
  9. Sim-FSVGD
  10. Illustrative Example: the Pendulum
  11. Main Algorithm
  12. Domain-model process
  13. Sim-to-real prior
  14. Estimating the Stochastic Process Score
  15. Experiments
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Dynamic RC car from our experiments.
  • Figure 2: BNN posteriors trained on two data points for a one-dimensional sinusoidal function. GreyBox finds accurate mean, but has large uncertainty estimates. At the same time Sim-FSVGD obtains both accurate mean and uncertainty estimates.
  • Figure 3: We benchmark NLL of dynamics model on test data for Racecar on hardware and in simulations and Pendulum in simulation. BNNs with simulation priors achieve low NLL already in small data regimes and constantly outperform the standard BNNs when we increase the number of train data points.
  • Figure 4: Desired reverse parking maneuver which involves rotating the car 180° and parking ca. 2m away.
  • Figure 5: Realized trajectory for Racecar environment in simulation after episode 3. In violet, we have the trajectory of the model with FSVGD (no prior), and in red the trajectory of Sim-FSVGD using the low-fidelity prior. We observe that, Sim-FSVGD already learns to get close to the target in three episodes, whereas FSVGD (no prior) explores away from it.
  • ...and 2 more figures