Safe MPC Alignment with Human Directional Feedback

TL;DR

The paper tackles learning safety constraints for model predictive control in robotics by enabling online, human directional feedback to shape a learnable constraint. It introduces Safe MPC Alignment, a certifiable method that updates the constraint by cutting the hypothesis space upon each directional correction, with convergence guarantees and misspecification certification. The approach uses a penalty-based Safe MPC, a linear-in-parameters safety function, and centers of maximum volume ellipsoids to drive efficient learning. Extensive simulations, user studies, and a real-world Franka arm experiment demonstrate data-efficient learning (tens of corrections) and robust performance across tasks, highlighting practical impact for safe robot-AI systems.

Abstract

In safety-critical robot planning or control, manually specifying safety constraints or learning them from demonstrations can be challenging. In this article, we propose a certifiable alignment method for a robot to learn a safety constraint in its model predictive control (MPC) policy from human online directional feedback. To our knowledge, it is the first method to learn safety constraints from human feedback. The proposed method is based on an empirical observation: human directional feedback, when available, tends to guide the robot toward safer regions. The method only requires the direction of human feedback to update the learning hypothesis space. It is certifiable, providing an upper bound on the total number of human feedback in the case of successful learning, or declaring the hypothesis misspecification, i.e., the true safety constraint cannot be found within the specified hypothesis space. We evaluated the proposed method in numerical examples and user studies with two simulation games. Additionally, we tested the proposed method on a real-world Franka robot arm performing mobile water-pouring tasks. The results demonstrate the efficacy and efficiency of our method, showing that it enables a robot to successfully learn safety constraints with a small handful (tens) of human directional corrections.

Figures (23)

• Figure 1: Illustration of the proposed safe MPC alignment. Our method online updates the safety constraint using human directional feedback, such that the robot eventually aligns the safe MPC policy with human intent for the safety-critical task.
• Figure 2: Update of Safe MPC policy with human directional corrections.
• Figure 3: Illustration of directional correction in the space of robot plan $\boldsymbol \xi$s. The grey circles represent the level contours of $B(\boldsymbol \xi, \boldsymbol \theta_H)$. The black dot stands for the current motion plan $\boldsymbol\xi_{\boldsymbol \theta}^\gamma$ and the red point stands for optimal plan $\boldsymbol\xi_{\boldsymbol{\theta}_H}^\gamma$ with $\theta_H$. The blue region stands for the half-space of all possible human correction under our assumption (\ref{['equ.core-neq']}). Some samples of human corrections are shown by blue arrows.
• Figure 4: The update of hypothesis space (2D example). Two cutting lines are used to cut $\Theta_{i-1}$ to $\Theta_i$. The user intent set $\bar{\Theta}_H$ is contained in $\Theta_i$. New $\boldsymbol \theta_{i+1}$ is chosen from $\Theta_i$ for the updated Safe MPC policy.
• Figure 5: Illustration of the different choices of $\boldsymbol \theta_{i}$. (a) and (b): When $\boldsymbol \theta_{i}$ is chosen near the set boundary, the removed volume depends on the specific direction of $\mathbf{bd}\ \mathcal{C}_{i}$. (c): When $\boldsymbol \theta_{i}$ is near the center, the cut volume is large on average regardless of the specific user directional correction.

Theorems & Definitions (20)

• Lemma 1
• Lemma 2
• proof
• Lemma 3
• proof
• Lemma 4
• proof
• Definition 1: Maximum Volume inscribed Ellipsoid (MVE)boyd2004convex
• Lemma 5
• proof