Interpretable and Explainable Machine Learning Methods for Predictive Process Monitoring: A Systematic Literature Review

TL;DR

This systematic literature review addresses the challenge of understanding and applying explainable and interpretable ML in predictive process monitoring. Using PRISMA, it synthesizes 67 studies to distinguish intrinsically interpretable approaches from post-hoc explanations, maps application domains and datasets (notably BPIC), and analyzes evaluation practices. Key findings include a dominance of interpretability in some methods (e.g., decision trees) and a heavy reliance on post-hoc XAI for black-box models, with gaps in formal evaluation and real-world deployment. The work provides a structured foundation for practitioners and researchers to design transparent predictive process analytics and to establish more rigorous, multidimensional evaluation frameworks.

Abstract

This paper presents a systematic literature review (SLR) on the explainability and interpretability of machine learning (ML) models within the context of predictive process mining, using the PRISMA framework. Given the rapid advancement of artificial intelligence (AI) and ML systems, understanding the "black-box" nature of these technologies has become increasingly critical. Focusing specifically on the domain of process mining, this paper delves into the challenges of interpreting ML models trained with complex business process data. We differentiate between intrinsically interpretable models and those that require post-hoc explanation techniques, providing a comprehensive overview of the current methodologies and their applications across various application domains. Through a rigorous bibliographic analysis, this research offers a detailed synthesis of the state of explainability and interpretability in predictive process mining, identifying key trends, challenges, and future directions. Our findings aim to equip researchers and practitioners with a deeper understanding of how to develop and implement more trustworthy, transparent, and effective intelligent systems for predictive process analytics.

Figures (17)

• Figure 1: Sources of input data accumulated in an event log and predictands of supervised learning DFKI_Smart_Lego_Factory
• Figure 2: Template for the analysis approach of retrieved literature.
• Figure 3: Flowchart depicting the retrieval and selection of retrieved publications, following the PRISMA approach.
• Figure 4: Number of identified publications per publication outlet.
• Figure 5: Number of identified publications per publication outlet grouped by year of publication.

Theorems & Definitions (13)

• Definition 1: Event
• Definition 2: Trace, Partial Trace, Prefix and Suffix
• Definition 3: Event Log
• Definition 4: Feature Extraction
• Definition 5: Labeling
• Definition 6: Supervised Learning
• Definition 7: Process Outcome Prediction
• Definition 8: Next Event Prediction
• Definition 9: Process Performance Indicator (PPI) Prediction
• Definition 10: Intrinsically Interpretable Model