Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AVOCADO: The Streaming Process Mining Challenge

Christian Imenkamp, Andrea Maldonado, Hendrik Reiter, Martin Werner, Wilhelm Hasselbring, Agnes Koschmider, Andrea Burattin

TL;DR

AVOCADO addresses the need for reproducible evaluation of streaming process mining algorithms under real-time constraints. It defines a standardized framework that separates problem and solution layers, and uses synthetic, drift-structured event streams to enable objective benchmarking. The evaluation hinges on online conformance metrics—Accuracy, MAE, RMSE, Processing Latency, and Robustness—and a weighted aggregate score to rank approaches, with a simple Frequency-Based Conformance baseline as a sanity check. By inviting the community to contribute challenges, metrics, and real-world complexities, the work aims to accelerate development of real-time process discovery and conformance techniques in streaming settings, formalized through a CPU-only, web-based submission pipeline and a configurable scoring scheme: $score = (0.3 × accuracy) + (0.25 × MAE) + (0.2 × RMSE) + (0.15 × latency) + (0.1 × robustness)$.

Abstract

Streaming process mining deals with the real-time analysis of streaming data. Event streams require algorithms capable of processing data incrementally. To systematically address the complexities of this domain, we propose AVOCADO, a standardized challenge framework that provides clear structural divisions: separating the concept and instantiation layers of challenges in streaming process mining for algorithm evaluation. The AVOCADO evaluates algorithms on streaming-specific metrics like accuracy, Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Processing Latency, and robustness. This initiative seeks to foster innovation and community-driven discussions to advance the field of streaming process mining. We present this framework as a foundation and invite the community to contribute to its evolution by suggesting new challenges, such as integrating metrics for system throughput and memory consumption, and expanding the scope to address real-world stream complexities like out-of-order event arrival.

AVOCADO: The Streaming Process Mining Challenge

TL;DR

AVOCADO addresses the need for reproducible evaluation of streaming process mining algorithms under real-time constraints. It defines a standardized framework that separates problem and solution layers, and uses synthetic, drift-structured event streams to enable objective benchmarking. The evaluation hinges on online conformance metrics—Accuracy, MAE, RMSE, Processing Latency, and Robustness—and a weighted aggregate score to rank approaches, with a simple Frequency-Based Conformance baseline as a sanity check. By inviting the community to contribute challenges, metrics, and real-world complexities, the work aims to accelerate development of real-time process discovery and conformance techniques in streaming settings, formalized through a CPU-only, web-based submission pipeline and a configurable scoring scheme: .

Abstract

Streaming process mining deals with the real-time analysis of streaming data. Event streams require algorithms capable of processing data incrementally. To systematically address the complexities of this domain, we propose AVOCADO, a standardized challenge framework that provides clear structural divisions: separating the concept and instantiation layers of challenges in streaming process mining for algorithm evaluation. The AVOCADO evaluates algorithms on streaming-specific metrics like accuracy, Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Processing Latency, and robustness. This initiative seeks to foster innovation and community-driven discussions to advance the field of streaming process mining. We present this framework as a foundation and invite the community to contribute to its evolution by suggesting new challenges, such as integrating metrics for system throughput and memory consumption, and expanding the scope to address real-world stream complexities like out-of-order event arrival.
Paper Structure (12 sections, 2 equations, 4 figures, 1 algorithm)

This paper contains 12 sections, 2 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Layer
  4. Process Discovery Contest
  5. Solution Layer
  6. Conceptualization of Streaming Process Mining
  7. Streaming Process Mining Challenge
  8. Algorithm Requirements
  9. Participation
  10. Parameters and Metrics
  11. Assessment of Requirements
  12. Conclusion

Figures (4)

  • Figure 1: AVOCADO: Unveiling the problem's 'pulp' to expose the solution set 'seed', conceived in the formalization of SPM and embodied in this challenge.
  • Figure 2: Conceptualization of the "rediscovery problem" in the context of Streaming Process Mining.
  • Figure 3: High-Level overview of the steps required to participate in the challenge
  • Figure 4: Case-Study to assess the requirements of AVOCADObased on the frequency-based conformance algorithm