Does Optimal Control Always Benefit from Better Prediction? An Analysis Framework for Predictive Optimal Control

Abstract

The ``prediction + optimal control'' scheme has shown good performance in many applications of automotive, traffic, robot, and building control. In practice, the prediction results are simply considered correct in the optimal control design process. However, in reality, these predictions may never be perfect. Under a conventional stochastic optimal control formulation, it is difficult to answer questions like ``what if the predictions are wrong''. This paper presents an analysis framework for predictive optimal control where the subjective belief about the future is no longer considered perfect. A novel concept called the hidden prediction state is proposed to establish connections among the predictors, the subjective beliefs, the control policies and the objective control performance. Based on this framework, the predictor evaluation problem is analyzed. Three commonly-used predictor evaluation measures, including the mean squared error, the regret and the log-likelihood, are considered. It is shown that neither using the mean square error nor using the likelihood can guarantee a monotonic relationship between the predictor error and the optimal control cost. To guarantee control cost improvement, it is suggested the predictor should be evaluated with the control performance, e.g., using the optimal control cost or the regret to evaluate predictors. Numerical examples and examples from automotive applications with real-world driving data are provided to illustrate the ideas and the results.

Figures (8)

• Figure 1: We sometimes assume a monotonic relationship like (a) between the predictor error and the optimal control cost. However, it is possible that the relationship is like (b), which means when the predictor improves, the control cost may get worse.
• Figure 2: A typical predictive optimal control structure. A predictor generates a belief of the future based on observations of the environment. The optimal controller decides the control input based on the belief.
• Figure 3: The hidden prediction state $s$ in the environment model. $s$ contains all factors from the environment that impacting the predictive optimal control process. It completely determines the realized disturbance sequence $\bar{w}$ and the observation sequence $\bar{o}$.
• Figure 4: Three typical types of recurrent predictions. (a) Type I: the only observation is obtained at step 0. (b) Type II: a new observation is obtained at every step. (c). Type III: the prediction covers a fixed-length receding horizon with new observations at every step.
• Figure 5: Impacters of the actual cost in predictive optimal control, when the predictor is given and the policy is optimal with respect to the belief. (a) In Type I, the actual cost is uniquely determined by $s$ through two parallel paths: the controller path (the observation-belief-policy path) and the physics path (the disturbance path). (b) Paths in Type II are similar to Type I. (c) In Type III, the actual cost is determined by $s$ and the artificial terminal cost in the controller path.

Theorems & Definitions (4)

• Definition 1: Belief
• Definition 2: Predictor
• Definition 3: Maximum Indistinguishable Observation Set
• Definition 4: Accurate