Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safety and optimality in learning-based control at low computational cost

Dominik Baumann, Krzysztof Kowalczyk, Cristian R. Rojas, Koen Tiels, Pawel Wachel

TL;DR

This paper tackles safe exploration in learning-based control with limited computation. It introduces CoLSafe, a kernel-based safe-learning method using the Nadaraya-Watson estimator that achieves sublinear data growth and provides high-probability safety and near-optimality guarantees, unlike GP-based SafeOpt which incurs cubic scaling. Theoretical results establish bounds on estimation error and uncertainty, and the algorithm maintains confidence intervals, a safe set, and exploration sets to ensure safety while improving reward. Empirical evaluation on a 7-DOF Franka robot arm shows CoLSafe can expand the safe set rapidly and attain near-optimal policies with substantially lower compute times both in simulation and hardware experiments. The work demonstrates a practical, scalable approach to safety-critical learning in robotics, with potential for embedding on resource-constrained platforms.

Abstract

Applying machine learning methods to physical systems that are supposed to act in the real world requires providing safety guarantees. However, methods that include such guarantees often come at a high computational cost, making them inapplicable to large datasets and embedded devices with low computational power. In this paper, we propose CoLSafe, a computationally lightweight safe learning algorithm whose computational complexity grows sublinearly with the number of data points. We derive both safety and optimality guarantees and showcase the effectiveness of our algorithm on a seven-degrees-of-freedom robot arm.

Safety and optimality in learning-based control at low computational cost

TL;DR

This paper tackles safe exploration in learning-based control with limited computation. It introduces CoLSafe, a kernel-based safe-learning method using the Nadaraya-Watson estimator that achieves sublinear data growth and provides high-probability safety and near-optimality guarantees, unlike GP-based SafeOpt which incurs cubic scaling. Theoretical results establish bounds on estimation error and uncertainty, and the algorithm maintains confidence intervals, a safe set, and exploration sets to ensure safety while improving reward. Empirical evaluation on a 7-DOF Franka robot arm shows CoLSafe can expand the safe set rapidly and attain near-optimal policies with substantially lower compute times both in simulation and hardware experiments. The work demonstrates a practical, scalable approach to safety-critical learning in robotics, with potential for embedding on resource-constrained platforms.

Abstract

Applying machine learning methods to physical systems that are supposed to act in the real world requires providing safety guarantees. However, methods that include such guarantees often come at a high computational cost, making them inapplicable to large datasets and embedded devices with low computational power. In this paper, we propose CoLSafe, a computationally lightweight safe learning algorithm whose computational complexity grows sublinearly with the number of data points. We derive both safety and optimality guarantees and showcase the effectiveness of our algorithm on a seven-degrees-of-freedom robot arm.
Paper Structure (18 sections, 14 theorems, 61 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 18 sections, 14 theorems, 61 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem setting
  4. Safe and optimal policy learning
  5. Nadaraya-Watson estimator
  6. The algorithm
  7. Evaluation
  8. Setup
  9. Simulations
  10. Hardware experiments
  11. Conclusion
  12. Proofs
  13. Proof of Lemma \ref{['lem:bounds']}
  14. Proof of Lemma \ref{['lem:bound_uncertainty']}
  15. Proof of Theorem \ref{['thm:safety']}
  16. ...and 3 more sections

Key Result

Lemma 1

Consider Assumptions ass:safe_seed-ass:kernel, where the bandwidth parameter $\lambda$ is a fixed constant. Assume that $|\mathcal{A}|\le D$. Then, for every $0 < \delta <1$, with probability larger than $1 - \delta$, for all $n \in \mathbb{N}$ and $a \in \mathcal{A}$, where

Figures (5)

  • Figure 1: Exploration behavior comparison of CoLSafe and SafeOpt. We can see that CoLSafe explores the safe set significantly faster. The yellow regions mark the safe set, and the blue triangle is the current optimum.
  • Figure 2: Performance of CoLSafe and SafeOpt on the simulated Franka robot. Both algorithms can similarly explore the safe region, while the computational footprint of CoLSafe is significantly lower.
  • Figure 3: Time complexity of SafeOpt and CoLSafe. We measure the time $t$ for updating the safe set and suggesting the next candidate point in each iteration $n$. For higher iterations, i.e., more data, SafeOpt requires significantly more time. Note the logarithmic scaling of the $y$-axis.
  • Figure 4: Ablation study. Believed optima over number of iterations for different kernels and bandwidth parameters $\lambda$.
  • Figure 5: CoLSafe vs. SafeOpt in real experiments. We show the reward achieved by the currently believed optimal parameters after varying number of iterations. We can see that CoLSafe can increase the reward faster than SafeOpt.

Theorems & Definitions (26)

  • Remark 1
  • Remark 2
  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Remark 3
  • Lemma 3
  • proof
  • Corollary 1
  • proof
  • ...and 16 more