Uncertainty measurement for complex event prediction in safety-critical systems

TL;DR

This paper proposes an approach incorporating ML and sensitivity analysis that verifies how the output varies according to each input parameter and measures the uncertainty associated with the predicted complex event.

Abstract

Complex events originate from other primitive events combined according to defined patterns and rules. Instead of using specialists' manual work to compose the model rules, we use machine learning (ML) to self-define these patterns and regulations based on incoming input data to produce the desired complex event. Complex events processing (CEP) uncertainty is critical for embedded and safety-critical systems. This paper exemplifies how we can measure uncertainty for the perception and prediction of events, encompassing embedded systems that can also be critical to safety. Then, we propose an approach (ML\_CP) incorporating ML and sensitivity analysis that verifies how the output varies according to each input parameter. Furthermore, our model also measures the uncertainty associated with the predicted complex event. Therefore, we use conformal prediction to build prediction intervals, as the model itself has uncertainties, and the data has noise. Also, we tested our approach with classification (binary and multi-level) and regression problems test cases. Finally, we present and discuss our results, which are very promising within our field of research and work.

Figures (7)

• Figure 1: Conformal predictor workflow
• Figure 2: Comparison of the traffic network used in the simulation with the real map of Toronto
• Figure 3: Proposed approach to estimate uncertainty in events
• Figure 4: Sobol analysis to indicate priority input for binary classification problem
• Figure 5: Sobol analysis to indicate priority input for multi-level classification problem