Neural Scaling Laws in Robotics

TL;DR

This work addresses whether neural scaling laws seen in language and vision apply to robotics, and to what extent data size, model size, and compute govern Robot Foundation Models and LLMs. It employs a large-scale meta-analysis of 327 robotics papers to fit power-law relations and quantify scaling exponents for data, model size, and compute, finding that robotic performance generally improves with scale and that data scaling is especially prominent. The study also reports emergent capabilities that arise as models scale, and discusses implications for benchmarking, efficiency, and safety in deploying scalable robot systems. Overall, the results suggest that while current robotic performance is moderate, expanding data and compute resources could yield substantial future gains, with scaling laws providing a framework for resource planning and capability forecasting.

Abstract

Neural scaling laws have driven significant advancements in machine learning, particularly in domains like language modeling and computer vision. However, the exploration of neural scaling laws within robotics has remained relatively underexplored, despite the growing adoption of foundation models in this field. This paper represents the first comprehensive study to quantify neural scaling laws for Robot Foundation Models (RFMs) and Large Language Models (LLMs) in robotics tasks. Through a meta-analysis of 327 research papers, we investigate how data size, model size, and compute resources influence downstream performance across a diverse set of robotic tasks. Consistent with previous scaling law research, our results reveal that the performance of robotic models improves with increased resources, following a power-law relationship. Promisingly, the improvement in robotic task performance scales notably faster than language tasks. This suggests that, while performance on downstream robotic tasks today is often moderate-to-poor, increased data and compute are likely to signficantly improve performance in the future. Also consistent with previous scaling law research, we also observe the emergence of new robot capabilities as models scale.

Figures (8)

• Figure 1: Growth of Robotics (a) and Scaling Laws (b) research over time
• Figure 2: Publication by sector (a) and top 10 research institutions by publications.
• Figure 3: Scaling laws in robotics: (a, c, e) show scaling across data, model size, and compute, with fitted power laws. (b, d, f) illustrates the relative scaling behaviors as $D$, $N$, and $C$ increase, modeled by $\text{Error Rate}(X) = X^{(\bar{\sigma_X} \pm K_X)}$, where $X$ denotes the input variable ($D$, $N$, or $C$), $\bar{\sigma_X}$ is the mean scaling exponent ($\alpha$, $\beta$, or $\gamma$), and $K_X$ represents the Confidence Interval of $\sigma_X$.
• Figure 4: Data size scaling laws with error bars (mean, standard error). Statistical significance is indicated as follows: n.s. (not significant) $p > 0.05$; * $p \leq 0.05$; ** $p \leq 0.01$; *** $p \leq 0.001$.
• Figure 5: Model size scaling laws with error bars (mean, standard error). Statistical significance is indicated as follows: n.s. (not significant) $p > 0.05$; * $p \leq 0.05$; ** $p \leq 0.01$; *** $p \leq 0.001$.