Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs

Alessandro Padella, Massimiliano de Leoni, Marlon Dumas

TL;DR

This work tackles predictive process monitoring in data-scarce environments by introducing an LLM-based prompting framework that encodes traces into text and prompts the model to predict KPIs with accompanying reasoning. It advances three key aspects: (i) a sequential trace encoding ($\rho_{seq}$) tailored for LLMs, (ii) a context-based seven-part prompting scheme that elicits predictions and explanations, and (iii) a systematic analysis of the LLM’s reasoning via manually extracted $\beta$-learners and Good‑Turing validation. Empirical results across three real logs for Total Time and Activity Occurrence show the LLM can outperform state-of-the-art baselines with as few as 100 traces, while semantic hashing confirms the model leverages embodied process knowledge. The study also demonstrates that the LLM’s reasoning patterns can be re-implemented as interpretable predictors, though they typically underperform the original LLM, underscoring the model’s richer inferential capabilities. Overall, the framework highlights the viability of LLMs for PPM in data-scarce settings and points to future work in extending prompting strategies and moving toward Prescriptive Process Analytics.

Abstract

Predictive Process Monitoring is a branch of process mining that aims to predict the outcome of an ongoing process. Recently, it leveraged machine-and-deep learning architectures. In this paper, we extend our prior LLM-based Predictive Process Monitoring framework, which was initially focused on total time prediction via prompting. The extension consists of comprehensively evaluating its generality, semantic leverage, and reasoning mechanisms, also across multiple Key Performance Indicators. Empirical evaluations conducted on three distinct event logs and across the Key Performance Indicators of Total Time and Activity Occurrence prediction indicate that, in data-scarce settings with only 100 traces, the LLM surpasses the benchmark methods. Furthermore, the experiments also show that the LLM exploits both its embodied prior knowledge and the internal correlations among training traces. Finally, we examine the reasoning strategies employed by the model, demonstrating that the LLM does not merely replicate existing predictive methods but performs higher-order reasoning to generate the predictions.

Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs

TL;DR

This work tackles predictive process monitoring in data-scarce environments by introducing an LLM-based prompting framework that encodes traces into text and prompts the model to predict KPIs with accompanying reasoning. It advances three key aspects: (i) a sequential trace encoding () tailored for LLMs, (ii) a context-based seven-part prompting scheme that elicits predictions and explanations, and (iii) a systematic analysis of the LLM’s reasoning via manually extracted -learners and Good‑Turing validation. Empirical results across three real logs for Total Time and Activity Occurrence show the LLM can outperform state-of-the-art baselines with as few as 100 traces, while semantic hashing confirms the model leverages embodied process knowledge. The study also demonstrates that the LLM’s reasoning patterns can be re-implemented as interpretable predictors, though they typically underperform the original LLM, underscoring the model’s richer inferential capabilities. Overall, the framework highlights the viability of LLMs for PPM in data-scarce settings and points to future work in extending prompting strategies and moving toward Prescriptive Process Analytics.

Abstract

Predictive Process Monitoring is a branch of process mining that aims to predict the outcome of an ongoing process. Recently, it leveraged machine-and-deep learning architectures. In this paper, we extend our prior LLM-based Predictive Process Monitoring framework, which was initially focused on total time prediction via prompting. The extension consists of comprehensively evaluating its generality, semantic leverage, and reasoning mechanisms, also across multiple Key Performance Indicators. Empirical evaluations conducted on three distinct event logs and across the Key Performance Indicators of Total Time and Activity Occurrence prediction indicate that, in data-scarce settings with only 100 traces, the LLM surpasses the benchmark methods. Furthermore, the experiments also show that the LLM exploits both its embodied prior knowledge and the internal correlations among training traces. Finally, we examine the reasoning strategies employed by the model, demonstrating that the LLM does not merely replicate existing predictive methods but performs higher-order reasoning to generate the predictions.
Paper Structure (19 sections, 5 equations, 4 figures, 7 tables)

This paper contains 19 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Approach For LLM-based Predictions
  5. An Encoding Function for LLMs
  6. Context-Based Prompting Technique for LLMs in Predictive Process Monitoring
  7. Example of Reasoning to Achieve Predictions
  8. Experimental Results
  9. LLM used and Train and Test Split
  10. Use Cases and Benchmarks
  11. Experimental Pipeline
  12. Prediction Quality Comparison
  13. Hashing of the Input Prompts
  14. Derivation and Re-implementation of LLM's $\beta$-learners
  15. Results
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Example of an encoding function applied to a running trace used for predicting the KPI “Total Time”.
  • Figure 2: Pipeline outlining the proposed method using LLMs for PPM, in which the KPI is “Total Time”.
  • Figure 3: Example of a usage of the Sequential encoding function $\rho_{aggr}$ applied to the same running trace and KPI employed in Figure \ref{['fig:encoding']}.
  • Figure 4: KPI convergences of various use cases.

Theorems & Definitions (2)

  • definition 1: Events
  • definition 2: Traces & Event Logs