Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prediction with Missing Data: Target Probabilities and Missingness Mechanisms

Pierre Catoire, Robin Genuer, Cecile Proust-Lima

Abstract

Conditions ensuring optimal parameter estimation in the presence of missing data are well established in inference, typically relying on the Missing-at-Random (MAR) assumption. In prediction, similar principles are often assumed to apply. However, methods considered biased in inference, such as pattern sub-modelling or unconditional imputation, have been shown to achieve optimal predictive performance under any missingness mechanism, including non-MAR (MNAR). To explain this apparent contradiction, we introduce a new formal framework for describing missingness in prediction. Central to this framework is a distinction between two prediction targets, defined according to whether or not the indicator of observation of the predictors is exploited to predict the outcome. This distinction leads to a classification of the missingness mechanisms describing the conditions under which these targets are equal, and when consistent prediction of each is achievable. A key result is that both targets may be consistently predicted under conditions weaker than MAR. We discuss the implications of this paradigm for handling missing data in prediction, distinguishing between missingness at development, validation and deployment of a forecaster. The findings are illustrated using simulated data and a real-world application with the prediction of significant injury after trauma upon arrival at the emergency department.

Prediction with Missing Data: Target Probabilities and Missingness Mechanisms

Abstract

Conditions ensuring optimal parameter estimation in the presence of missing data are well established in inference, typically relying on the Missing-at-Random (MAR) assumption. In prediction, similar principles are often assumed to apply. However, methods considered biased in inference, such as pattern sub-modelling or unconditional imputation, have been shown to achieve optimal predictive performance under any missingness mechanism, including non-MAR (MNAR). To explain this apparent contradiction, we introduce a new formal framework for describing missingness in prediction. Central to this framework is a distinction between two prediction targets, defined according to whether or not the indicator of observation of the predictors is exploited to predict the outcome. This distinction leads to a classification of the missingness mechanisms describing the conditions under which these targets are equal, and when consistent prediction of each is achievable. A key result is that both targets may be consistently predicted under conditions weaker than MAR. We discuss the implications of this paradigm for handling missing data in prediction, distinguishing between missingness at development, validation and deployment of a forecaster. The findings are illustrated using simulated data and a real-world application with the prediction of significant injury after trauma upon arrival at the emergency department.
Paper Structure (62 sections, 59 equations, 9 figures, 3 tables)

This paper contains 62 sections, 59 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Notations and definitions for inference and prediction
  3. Initial notations
  4. Missing values in inference
  5. Missing values in prediction
  6. Forecasters and training procedures
  7. Optimality in prediction
  8. Role of the MAR assumption in prediction
  9. A nomenclature for probability distribution targets and missingness mechanisms for prediction
  10. What is to predict? Missingness-Unconditional and Missingness-Conditional probabilities
  11. Nomenclature of missingness mechanisms for prediction
  12. Informative missingness of the predictors for the outcome
  13. Missingness at Random for the predictors
  14. Sufficient conditions for Bayes consistent procedures
  15. Bayes consistent training procedures for MU risk
  16. ...and 47 more sections

Figures (9)

  • Figure 1: Relations of implication between Missingness Completely at Random (MCAR), Missingness at Random (MAR), MAR for predictors - outcome missing (MARX-YM) and observed (MARX-YO), Non-Informative Missingness of the predictors for the outcome (NIMO) and Non-informative Complete Observation of the predictors for the outcome (NICO). Bold arrows represent direct implications. Dashed arrows represent joint implications.
  • Figure 2: Directed Acyclic Graphs of five illustrative scenarios with two predictors and one outcome, and their missingness mechanism properties. Bold and dashed nodes represent model variables and missingness indicators, respectively.
  • Figure 3: Mean Squared Error (MSE) for various proportions of missing values of evaluated procedures for the five scenarios, for continuous variables
  • Figure 4: Distribution of the observation patterns. From naniar package tierney_naniar_2021.
  • Figure : Table S1. Illustrative example of a MAR missingness scenario without MARX-YM, NICO nor NIMO
  • ...and 4 more figures