All AI Models are Wrong, but Some are Optimal

TL;DR

This work tackles the gap between predictive accuracy and decision quality in sequential decision-making by introducing decision-oriented predictive models and proving necessary and sufficient conditions for when such models yield optimal policies. It shows that models best fitting data do not necessarily maximize decision performance and that deterministic predictions can be optimal for stochastic systems under the proposed framework. Through battery energy storage and smart-home heat-pump examples, the authors demonstrate how RL-based fine-tuning of data-fitted predictors can materially improve closed-loop performance, even when using simple, differentiable MPC schemes. The results offer practical guidance for constructing decision-oriented predictive models and highlight future directions for integrating prediction, optimization, and learning in complex, real-world systems.

Abstract

AI models that predict the future behavior of a system (a.k.a. predictive AI models) are central to intelligent decision-making. However, decision-making using predictive AI models often results in suboptimal performance. This is primarily because AI models are typically constructed to best fit the data, and hence to predict the most likely future rather than to enable high-performance decision-making. The hope that such prediction enables high-performance decisions is neither guaranteed in theory nor established in practice. In fact, there is increasing empirical evidence that predictive models must be tailored to decision-making objectives for performance. In this paper, we establish formal (necessary and sufficient) conditions that a predictive model (AI-based or not) must satisfy for a decision-making policy established using that model to be optimal. We then discuss their implications for building predictive AI models for sequential decision-making.

Figures (6)

• Figure 1: Model-based decision-making concept presented with the example of an indoor vertical farm. The standard model-based decision-making framework involves estimating a predictive model from data to derive optimal decisions for the real system. Then this predictive model is used in silico to generate the best possible decisions for a given decision-making objective. The model-based decisions are then implemented on the real system.
• Figure 2: Expected-value and mle of a probability density function $\rho$.
• Figure 3: Constructing decision-oriented predictive models: The upper half (Phase 1) of the schematic illustrates the conventional approach to constructing predictive models, focusing on estimating the best fit to the real system's dynamics using the data and a choice of model estimation method. The lower half (Phase 2) represents the proposed approach to constructing decision-oriented predictive models using rl. The key difference lies in the estimation approach, where the decision objective is embedded into the estimation process together with the support of the conventional data-fitting approach.
• Figure 4: (Top row) Illustration of the energy storage example \ref{['sec:BatExample']}. The optimal value function ($V^\star$) and policy ($\pi^\star$) of the MDP are approximated by the mpc as $\hat{V}^\star$ and ${\hat{\pi}}^\star$, respectively. Although the value functions are nearly linear, the policies differ significantly due to the activation of the lower bound. (Bottom row) A similar illustration of the non-smooth value function in example \ref{['sec:AbsValExample']}.
• Figure 5: Illustration of the models obtained from the sufficient condition \ref{['eq:suficient_condition']} for different values of $\Delta$. The left plot shows the resulting value functions for the real mdp and the model-based mdp (mpc). The right side plot shows the models in terms of $\boldsymbol{\mathrm{s}}+\boldsymbol{\mathrm{a}}$. The solid curve is the expected value of the real state transition, and the other curves are the optimal models for different values of $\Delta$.

Theorems & Definitions (8)

• Definition 1
• Lemma 1
• Theorem 1
• Corollary 1
• Corollary 2
• Proposition 1
• Lemma 2
• Lemma 3