Experiment Design with Gaussian Process Regression with Applications to Chance-Constrained Control

TL;DR

This work focuses on the setting in which inference on the unknown dynamics is performed using Gaussian processes, and designs experiments via gradient descent on the expected control performance with respect to the experiment input on a chance-constrained minimum expected time control problem.

Abstract

Learning for control in repeated tasks allows for well-designed experiments to gather the most useful data. We consider the setting in which we use a data-driven controller that does not have access to the true system dynamics. Rather, the controller uses inferred dynamics based on the available information. In order to acquire data that is beneficial for this controller, we present an experimental design approach that leverages the current data to improve expected control performance. We focus on the setting in which inference on the unknown dynamics is performed using Gaussian processes. Gaussian processes not only provide uncertainty quantification but also allow us to leverage structures inherent to Gaussian random variables. Through this structure, we design experiments via gradient descent on the expected control performance with respect to the experiment input. In particular, we focus on a chance-constrained minimum expected time control problem. Numerical demonstrations of our approach indicate our experimental design outperforms relevant benchmarks.

Figures (4)

• Figure 1: Starting from the origin, the objective is to reach the green target set, $\mathcal{X}_s$, in minimum time. Overshooting the target set results in a crash in $\mathcal{X}_c$. The dynamics of the system are unknown and need to be learned.
• Figure 2: We visualize our GP approximation of the process mean and variance by taking slices of the input at three different levels, $u_{max}=0.1,0,u_{min}=-0.1$, and sweep the state. In blue, we observe the actual process with relatively small 1-$\sigma$ bounds from the process noise. The pre-experiment GP's dataset has 25 random trials, while the post-experiment GPs' datasets contain the original 25 plus the respective experiment trial. For $u_{max}$, we observe the GP models have relatively low epistemic uncertainty (variance), which is reasonable given the random walks have positive mean. Our experiment input, $\bar{U}$ reduces the uncertainty for zero-input significantly relative to the pre-experiment or $U^*$. The closed-loop, $U^*(x)$ is left off here for legibility. While $U^*$ appears to generate a model with less uncertainty for $u_{min}$, this is less relevant since the optimal control input does not need to be negative in this setup.
• Figure 3: We illustrate the optimal policies post-experiment. Since we have slightly different policies depending on the experiment outcome, we illustrate the policy closest to the median for our experiment design and the corresponding outcome for the benchmarks. We observe that the policy given perfect information is to apply the max input, 0.1, until the state reaches 0.6, then decrease until around 0.75, and then decrease the input to zero after just before 0.8. In this example, our experiment design $\bar{U}$ tends to approach the optimal without exceeding it. In some areas the benchmarks are closer to the perfect information policy but they also exceed it, which leads to riskier behavior.
• Figure 4: We visualize the benefit of adding a designed experiment to a dataset of random trials (e.g. 5 random trials plus one experiment) and test the control performance with this augmented dataset. Control feasibility indicates the fraction of controllers that satisfy the chance constraint over 1000 experiment outcomes. Of the controllers that satisfy the constraint, we can compute the expected finish time and 95 percent confidence intervals. Our experiment design, $\bar{U}$, outperforms all other benchmarks for control feasibility except for $U^*(x)$ for the dataset of size 10. Finishing time performance becomes more relevant as the feasibility metric nears 100 percent whereupon we see our time decreasing significantly relative to the others. Due to low control feasibility, we omit the finish time for the random case.

Theorems & Definitions (3)

• proof
• proof
• proof