Active Constraint Learning in High Dimensions from Demonstrations

TL;DR

The paper tackles unknown environmental constraint inference in high-dimensional robotic systems by introducing GP-ACL, an active constraint learning framework built on Gaussian process representations derived from KKT information in demonstrations. It iteratively selects start/goal constraint states using posterior GP samples to elicit informative demonstrations, thereby reducing epistemic uncertainty with a small, high-quality dataset. The approach advances constraint inference by explicitly targeting uncertainty reduction, and demonstrates superior constraint recovery accuracy across 2D/3D dynamics, high-dimensional simulations, and a 7-DOF robotic arm, compared with random sampling baselines. The work has practical implications for safe autonomous operation in complex, nonlinear environments, enabling more data-efficient learning of unknown constraints in continuous state spaces.

Abstract

We present an iterative active constraint learning (ACL) algorithm, within the learning from demonstrations (LfD) paradigm, which intelligently solicits informative demonstration trajectories for inferring an unknown constraint in the demonstrator's environment. Our approach iteratively trains a Gaussian process (GP) on the available demonstration dataset to represent the unknown constraints, uses the resulting GP posterior to query start/goal states, and generates informative demonstrations which are added to the dataset. Across simulation and hardware experiments using high-dimensional nonlinear dynamics and unknown nonlinear constraints, our method outperforms a baseline, random-sampling based method at accurately performing constraint inference from an iteratively generated set of sparse but informative demonstrations.

Figures (11)

• Figure 1: Given demonstrations (orange) generated via a 7-DOF robot arm, our GP-ACL algorithm recovers a nonlinear obstacle set with fewer instances of false positive (FP, in green) and false negative (FN, in red) errors, compared to a random-sampling baseline method.
• Figure 2: (a) Our method for finding start / goal constraint states that induce tight demonstrations. Blue dots represent tight points collected from demonstrations and used to train the GP constraint representation (see Sec. \ref{['subsec: Constraint Tightness and Constraint Gradient Extraction']}, \ref{['subsec: Training Gaussian Process (GP)-based Constraint Representations']}, and \ref{['subsec: App, Extraction of Tight Points and Constraint Gradient Estimates']}). To handle locally-convex constraint boundaries, we search along a hyperplane $\mathcal{H}(\eta)$ orthogonal to the gradient $\nabla \hat{g}_p(\kappa_{\text{MI},p})$. (b) To handle locally-concave constraint boundaries, we search along the gradient $\nabla \hat{g}_p(\kappa_{\text{MI},p})$. (c) Visualization of the locally-convex case in 3D.
• Figure 3: Our GP-ACL algorithm (left half) outperforms the random sampling baseline (right half) in accurately recovering constraints $g_{{\urcorner k},1}^\star$ (top), $g_{{\urcorner k},2}^\star$ (middle), and $g_{{\urcorner k},3}^\star$ (bottom) from unicycle dynamics demonstrations, with fewer false positive (green) and false negative (red) errors.
• Figure 4: Our GP-ACL algorithm (left half) outperforms the random sampling baseline (right half) in accurately recovering constraints $g_{{\urcorner k},3}^\star$ (top), $g_{{\urcorner k},7}^\star$ (middle/bottom), from unicycle dynamics (top) and simulated 7-DOF arm (middle/bottom) demonstrations, with fewer false positive (green) and false negative (red) errors. Middle and bottom row figures display 3D slices of the 7D constraint space from our numerical simulations on the 7-DOF arm.
• Figure 5: Constraint recovery accuracy of (a) our GP-ACL algorithm and (b) the random-sampling baseline approach. Across the following constraint learning tasks, our GP-ACL method consistently recovered the a priori unknown constraints more accurately compared to the random-sampling baseline method: Learning the constraints $g_{{\urcorner k},1}^\star$, $g_{{\urcorner k},2}^\star$, and $g_{{\urcorner k},3}^\star$ from demonstrations generated using unicycle dynamics (blue, red, and yellow, respectively); learning the constraint $g_{{\urcorner k},5}^\star$ from quadcopter dynamics (purple); and learning the constraint $g_{{\urcorner k},6}^\star$ from double integrator dynamics (green).

Theorems & Definitions (1)

• remark 1