Sample Efficient Robot Learning in Supervised Effect Prediction Tasks

TL;DR

Robots learning action–effect models in self-supervised settings incur high data collection costs; this work addresses this by proposing MUSEL, a model-uncertainty driven AL framework for regression in continuous spaces. MUSEL leverages a Stochastic Variational Deep Kernel Learning backbone to jointly estimate data and model uncertainty, learning progress (LP), and input diversity for sample selection. The main contributions are: (i) a concrete AL algorithm for continuous-state/action regression, (ii) a data uncertainty estimator that uses LP and minimum-distance, and (iii) experimental validation in one- and two-sphere tabletop tasks showing improved learning accuracy and sample efficiency with ablations. The results demonstrate practical gains for sample-efficient world-model learning in robotics and suggest broader applicability to self-supervised policy and control tasks.

Abstract

In self-supervised robotic learning, agents acquire data through active interaction with their environment, incurring costs such as energy use, human oversight, and experimental time. To mitigate these, sample-efficient exploration is essential. While intrinsic motivation (IM) methods like learning progress (LP) are widely used in robotics, and active learning (AL) is well established for classification in machine learning, few frameworks address continuous, high-dimensional regression tasks typical of world model learning. We propose MUSEL (Model Uncertainty for Sample-Efficient Learning), a novel AL framework tailored for regression tasks in robotics, such as action-effect prediction. MUSEL introduces a model uncertainty metric that combines total predictive uncertainty, learning progress, and input diversity to guide data acquisition. We validate our approach using a Stochastic Variational Deep Kernel Learning (SVDKL) model in two robotic tabletop tasks. Experimental results demonstrate that MUSEL improves both learning accuracy and sample efficiency, validating its effectiveness in learning action effects and selecting informative samples.

Figures (6)

• Figure 1: The proposed model, MUSEL is illustrated. See text for details.
• Figure 2: (a) One-sphere setting: The sphere is placed at the location selected by the robot prior to the push action. The blue region indicates allowed positions for the sphere. The diagonal wall is included to increase the task difficulty. (b) Two-sphere setting: This configuration is identical to (a) except for a second sphere, which moves according to physical dynamics but is set to a fixed location before action execution.
• Figure 3: Typical LP profiles of illustrative regions defined with $\alpha, pos_x, pos_y \in [-8.57, 8.57] \times [-0.43, 0.71) \times S$ are shown, where $S$ is indicated in the legend of each LP curve. For ease of comparison some LP curves are superimposed.
• Figure 4: Left: RMSE values obtained with single-measure-based and random sampling are contrasted with those of MUSEL as a function of sampling iterations. Error bars indicate the standard error of the mean (SEM). Right: Performance of MUSEL is contrasted with its performance when individual sampling components are ablated.
• Figure 5: Histogram of sample selection by random sampling (left) and MUSEL (right) over the space of $pos_x, pos_y$ with a grid of 50 x 50 over 10 repetitions. The histograms are normalized and scaled logarithmically for better visibility, and boundary of the regions in LP-based sampling are shown as white lines.