ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning

TL;DR

This work tackles the challenge of tuning black-box robotics simulators without tractable likelihoods by introducing Active Simulation-Based Inference (ASBI), which actively collects informative real-world data through robot actions. ASBI extends Neural Posterior Estimation to condition on both observations and actions, using a utility that approximates information gain to guide action selection in online rounds. Through sim-to-sim experiments on numerical models and robotics simulators, ASBI achieves faster convergence and sharper posteriors than baselines, and a real-to-sim experiment with beads and gravel demonstrates practical calibration benefits. The approach reduces the need for extensive offline data and enables efficient, targeted data collection, with potential for broader applicability in sim-to-real calibration and robotic system identification.

Abstract

Black-box simulators are widely used in robotics, but optimizing their parameters remains challenging due to inaccessible likelihoods. Simulation-Based Inference (SBI) tackles this issue using simulation-driven approaches, estimating the posterior from offline real observations and forward simulations. However, in black-box scenarios, preparing observations that contain sufficient information for parameter estimation is difficult due to the unknown relationship between parameters and observations. In this work, we present Active Simulation-Based Inference (ASBI), a parameter estimation framework that uses robots to actively collect real-world online data to achieve accurate black-box simulator tuning. Our framework optimizes robot actions to collect informative observations by maximizing information gain, which is defined as the expected reduction in Shannon entropy between the posterior and the prior. While calculating information gain requires the likelihood, which is inaccessible in black-box simulators, our method solves this problem by leveraging Neural Posterior Estimation (NPE), which leverages a neural network to learn the posterior estimator. Three simulation experiments quantitatively verify that our method achieves accurate parameter estimation, with posteriors sharply concentrated around the true parameters. Moreover, we show a practical application using a real robot to estimate the simulation parameters of cubic particles corresponding to two real objects, beads and gravel, with a bucket pouring action.

Figures (17)

• Figure 1: Particle-parameter estimation task with ASBI. A robot collects real-world informative data and then optimizes the simulation parameters to match the simulated data to the real data. By selecting the optimal action that maximizes Information Gain (IG) and gathering real-world observations online in each round, ASBI can achieve accurate parameter estimation with a small number of real executions.
• Figure 2: (a) Overview of ASBI. ASBI enables accurate parameter estimation for black-box simulators by selecting the optimal action using forward simulation and a learned posterior estimator. (b) Structure of posterior estimator. To approximate the posterior without the likelihood, ASBI trains a posterior estimator using a neural network that takes observation and action as input and outputs a set of parameters for $k$-MoGs.
• Figure 3: Information gain graphs and comparison of estimated posterior at different actions obtained using ASBI for the numerical model. Each row describes, for round $r$, the prior, the graph of information gain, and the posteriors obtained at different actions. Tested actions (a), (b), and (c) are shown with green diamonds on the information gain graph. The posteriors on optimal actions $(\xi^{*}_1, \xi^{*}_2, \xi^{*}_3) = (5.0, 0.5, 0.0)$ are highlighted in yellow, and each highlighted posterior is used as the prior of the next round. The first round with action(c), $\xi^{*}_1=5.0$, effectively reduces uncertainty in $\theta_2$ prediction. Then, in rounds 2 and 3, actions near zero reduce uncertainty in $\theta_1$ prediction. Using the informative observation obtained with the optimal action at each round, ASBI successfully estimates the posterior highly peaked around the true parameter $\boldsymbol{\theta}^{\text{true}} = (-3, 1)$, which is represented with a red star.
• Figure 4: Comparison of the changes in posteriors through 4-round parameter estimation using different methods for the numerical model. The true parameters are shown with a red star.
• Figure 5: Comparison of log-probability of ground-truth parameter at each round $r$ for the numerical model. Log-probability of the initial prior $p(\boldsymbol{\theta}) = 0.01$ is shown with a dotted purple line.