Complex Event Recognition with Symbolic Register Transducers: Extended Technical Report

TL;DR

The paper tackles the challenge of formally and efficiently performing Complex Event Recognition on streams by introducing Symbolic Register Transducers (SRT) and Symbolic Regular Expressions with Memory and Output (SREMO). SRT/SREMO provide denotational semantics, support n-ary relational constraints, and allow marking of input events that comprise complex events, enabling explicit enumeration of matches. The work establishes theoretical properties, including closure under core operators, and analyzes the impact of windowing on determinization and complementability, while providing a practical streaming CER engine with Wayeb and extensive experiments showing superior expressiveness and performance against state-of-the-art systems. It also discusses extensions to handle selection strategies, aggregates, and hierarchies, and outlines future directions for optimization and complex-event forecasting.

Abstract

We present a system for Complex Event Recognition (CER) based on automata. While multiple such systems have been described in the literature, they typically suffer from a lack of clear and denotational semantics, a limitation which often leads to confusion with respect to their expressive power. In order to address this issue, our system is based on an automaton model which is a combination of symbolic and register automata. We extend previous work on these types of automata, in order to construct a formalism with clear semantics and a corresponding automaton model whose properties can be formally investigated. We call such automata Symbolic Register Transducers (SRT). We show that SRT are closed under various operators, but are not in general closed under complement and they are not determinizable. However, they are closed under these operations when a window operator, quintessential in Complex Event Recognition, is used. We show how SRT can be used in CER in order to detect patterns upon streams of events, using our framework that provides declarative and compositional semantics, and that allows for a systematic treatment of such automata. For SRT to work in pattern detection, we allow them to mark events from the input stream as belonging to a complex event or not, hence the name "transducers". We also present an implementation of SRT which can perform CER. We compare our SRT-based CER engine against other state-of-the-art CER systems and show that it is both more expressive and more efficient.

Figures (9)

• Figure 1: $\mathit{SRT}$ corresponding to the $\mathit{SREMO}$ of eq. \ref{['srem:b_seq_s_filter_eq_id']}.
• Figure 2: A run of the $\mathit{SRT}$ of Figure \ref{['fig:example1']}, while consuming the first four events from the stream of Table \ref{['table:example_stream']}. Triggered transitions are shown in red and the current state of the $\mathit{SRT}$ in dark gray. The dashed box represents a register. The contents of the register at each configuration are shown inside the dashed box. Inside the dotted boxes, the run is shown.
• Figure 3: Constructing $\mathit{SRT}$ from $\mathit{SREMO}$\ref{['sremo:sremo2srt_example']}. New elements added at every step are shown in blue.
• Figure 5: Constructing a deterministic $\mathit{SRT}$ from the $\mathit{SRT}$ of Figure \ref{['fig:determinization_example_main']}.
• Figure 6: Throughput and memory consumption for sequential patterns with $n$-ary predicates as a function of pattern length. Window sizes are $w_{\mathit{stock}}=500$, $w_{\mathit{smart}}=5$, $w_{\mathit{taxi}}=100$.

Theorems & Definitions (53)

• Example 1
• Definition 2: $\mathcal{V}$-structure hedman2004first
• Example 3
• Definition 4: Term hedman2004first
• Definition 5: Formula hedman2004first
• Definition 6: $\mathcal{V}$-formula hedman2004first
• Example 7
• Definition 8: Model of $\mathcal{V}$-formulas hedman2004first
• Example 9
• Definition 10: Condition