Chameleons do not Forget: Prompt-Based Online Continual Learning for Next Activity Prediction

Abstract

Predictive process monitoring (PPM) focuses on predicting future process trajectories, including next activity predictions. This is crucial in dynamic environments where processes change or face uncertainty. However, current frameworks often assume a static environment, overlooking dynamic characteristics and concept drifts. This results in catastrophic forgetting, where training while focusing merely on new data distribution negatively impacts the performance on previously learned data distributions. Continual learning addresses, among others, the challenges related to mitigating catastrophic forgetting. This paper proposes a novel approach called Continual Next Activity Prediction with Prompts (CNAPwP), which adapts the DualPrompt algorithm for next activity prediction to improve accuracy and adaptability while mitigating catastrophic forgetting. We introduce new datasets with recurring concept drifts, alongside a task-specific forgetting metric that measures the prediction accuracy gap between initial occurrence and subsequent task occurrences. Extensive testing on three synthetic and two real-world datasets representing several setups of recurrent drifts shows that CNAPwP achieves SOTA or competitive results compared to five baselines, demonstrating its potential applicability in real-world scenarios. An open-source implementation of our method, together with the datasets and results, is available at: https://github.com/SvStraten/CNAPwP.

Figures (12)

• Figure 1: Catastrophic forgetting in sequential image classification. The model trains on realistic elephant photos (Task 1) and subsequently on bird drawings (Task 2), which typically causes it to forget the initial task. When Task 1 recurs as elephant drawings, performance drops, illustrating the need to leverage task-invariant features to preserve knowledge during shifts.
• Figure 2: Framework for Continual Next Activity Prediction with Prompts (CNAPwP). When concept drift is externally detected, a new task-specific E-Prompt is initialized. When no drifts are detected, a prompt detection mechanism dynamically selects and attaches the appropriate general G-Prompt and E-Prompt to the MHSA layers for accurate prediction.
• Figure 3: The accuracy per event for all methods on the ImbalancedTasks dataset. The plot reveals that while baselines like TFCLPM initially perform well, they suffer from degradation in later stages of the stream. In contrast, CNAPwP (dark blue line) demonstrates robust adaptation, maintaining higher stability and positive performance deltas even as task frequencies vary significantly throughout the stream.
• Figure 4: Heatmaps showing the accuracy drop per task (task-specific forgetting) for the ImbalancedTasks dataset for the four best-performing methods. Warm colors indicate positive knowledge retentions (improvements), white indicates stability and cold colors indicate forgetting.
• Figure 5: The accuracy per event for all methods on the RecurrentTasks dataset. It highlights CNAPwP's ability to recover accuracy swiftly following task transitions, whereas DynaTrainCDD takes noticeably longer to regain performance. The graph also exposes the instability of the IncrementalUpdate baseline (w = LastDrift), which suffers from deep, catastrophic spikes during specific task transitions due to insufficient training data.

Theorems & Definitions (9)

• definition 1: Next Activity Prediction
• definition 2: Online Next Activity Prediction
• definition 3: Learning Task
• definition 4: Task Incremental Learning
• definition 5: Concept Drift gama2014survey
• definition 6: Recurrent Concept Drifts
• definition 7: Catastrophic Forgetting kirkpatrick2017overcoming
• definition 8: Continual Learning kosciuszekonline
• definition 9: Continual Learning for Next Activity Prediction