Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Root Causing Prediction Anomalies Using Explainable AI

Ramanathan Vishnampet, Rajesh Shenoy, Jianhui Chen, Anuj Gupta

TL;DR

The paper tackles root-cause analysis of performance degradation in online-learning systems by identifying feature corruptions that propagate to labels and concepts. It introduces a local feature importance (LFI) estimation via feature ablation, then aggregates to global feature importances (GFI) and ranks features by shifts between control and anomaly windows. Compared with a model-feature correlation baseline (MFC), the XAI-based approach yields higher recall in locating corrupted features across 11 corruption scenarios, including embedding-related corruptions. The method is demonstrated on production-scale personalized advertising data with thousands of features and data pipelines, offering a cheap, model-agnostic monitoring tool and potential for self-healing with human oversight. These results support adopting explainability-driven monitoring to preserve data integrity and user experience in dynamic, continuously trained systems.

Abstract

This paper presents a novel application of explainable AI (XAI) for root-causing performance degradation in machine learning models that learn continuously from user engagement data. In such systems a single feature corruption can cause cascading feature, label and concept drifts. We have successfully applied this technique to improve the reliability of models used in personalized advertising. Performance degradation in such systems manifest as prediction anomalies in the models. These models are typically trained continuously using features that are produced by hundreds of real time data processing pipelines or derived from other upstream models. A failure in any of these pipelines or an instability in any of the upstream models can cause feature corruption, causing the model's predicted output to deviate from the actual output and the training data to become corrupted. The causal relationship between the features and the predicted output is complex, and root-causing is challenging due to the scale and dynamism of the system. We demonstrate how temporal shifts in the global feature importance distribution can effectively isolate the cause of a prediction anomaly, with better recall than model-to-feature correlation methods. The technique appears to be effective even when approximating the local feature importance using a simple perturbation-based method, and aggregating over a few thousand examples. We have found this technique to be a model-agnostic, cheap and effective way to monitor complex data pipelines in production and have deployed a system for continuously analyzing the global feature importance distribution of continuously trained models.

Root Causing Prediction Anomalies Using Explainable AI

TL;DR

The paper tackles root-cause analysis of performance degradation in online-learning systems by identifying feature corruptions that propagate to labels and concepts. It introduces a local feature importance (LFI) estimation via feature ablation, then aggregates to global feature importances (GFI) and ranks features by shifts between control and anomaly windows. Compared with a model-feature correlation baseline (MFC), the XAI-based approach yields higher recall in locating corrupted features across 11 corruption scenarios, including embedding-related corruptions. The method is demonstrated on production-scale personalized advertising data with thousands of features and data pipelines, offering a cheap, model-agnostic monitoring tool and potential for self-healing with human oversight. These results support adopting explainability-driven monitoring to preserve data integrity and user experience in dynamic, continuously trained systems.

Abstract

This paper presents a novel application of explainable AI (XAI) for root-causing performance degradation in machine learning models that learn continuously from user engagement data. In such systems a single feature corruption can cause cascading feature, label and concept drifts. We have successfully applied this technique to improve the reliability of models used in personalized advertising. Performance degradation in such systems manifest as prediction anomalies in the models. These models are typically trained continuously using features that are produced by hundreds of real time data processing pipelines or derived from other upstream models. A failure in any of these pipelines or an instability in any of the upstream models can cause feature corruption, causing the model's predicted output to deviate from the actual output and the training data to become corrupted. The causal relationship between the features and the predicted output is complex, and root-causing is challenging due to the scale and dynamism of the system. We demonstrate how temporal shifts in the global feature importance distribution can effectively isolate the cause of a prediction anomaly, with better recall than model-to-feature correlation methods. The technique appears to be effective even when approximating the local feature importance using a simple perturbation-based method, and aggregating over a few thousand examples. We have found this technique to be a model-agnostic, cheap and effective way to monitor complex data pipelines in production and have deployed a system for continuously analyzing the global feature importance distribution of continuously trained models.
Paper Structure (11 sections, 5 equations, 2 figures, 1 table)

This paper contains 11 sections, 5 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Statement
  4. Methodology
  5. Local Feature Importance Estimation
  6. Choice of Baseline
  7. Global Aggregation
  8. Model-Feature Correlation
  9. Results
  10. Conclusions
  11. Acknowledgments.

Figures (2)

  • Figure 1: Distribution of absolute values of LFIs for a particular feature using the definitions in \ref{['eq:prediction-lfi']} and \ref{['eq:pseudo-loss-lfi']}
  • Figure 2: Comparison of GFI distribution for control and anomaly data for a model with and without layer normalization. $F_1$ and $F_2$ are two sparse-encoded embedding features whose encoded representation is corrupted by dividing each value by $10$.