Learning to Predict, Discover, and Reason in High-Dimensional Discrete Event Sequences

Abstract

Electronic control units (ECUs) embedded within modern vehicles generate a large number of asynchronous events known as diagnostic trouble codes (DTCs). These discrete events form complex temporal sequences that reflect the evolving health of the vehicle's subsystems. In the automotive industry, domain experts manually group these codes into higher-level error patterns (EPs) using Boolean rules to characterize system faults and ensure safety. However, as vehicle complexity grows, this manual process becomes increasingly costly, error-prone, and difficult to scale. Notably, the number of unique DTCs in a modern vehicle is on the same order of magnitude as the vocabulary of a natural language, often numbering in the tens of thousands. This observation motivates a paradigm shift: treating diagnostic sequences as a language that can be modeled, predicted, and ultimately explained. Traditional statistical approaches fail to capture the rich dependencies and do not scale to high-dimensional datasets characterized by thousands of nodes, large sample sizes, and long sequence lengths. Specifically, the high cardinality of categorical event spaces in industrial logs poses a significant challenge, necessitating new machine learning architectures tailored to such event-driven systems. This thesis addresses automated fault diagnostics by unifying event sequence modeling, causal discovery, and large language models (LLMs) into a coherent framework for high-dimensional event streams. It is structured in three parts, reflecting a progressive transition from prediction to causal understanding and finally to reasoning for vehicle diagnostics. Consequently, we introduce several Transformer-based architectures for predictive maintenance, scalable sample- and population-level causal discovery frameworks and a multi-agent system that automates the synthesis of Boolean EP rules.

Figures (44)

• Figure 1: Error Pattern Prediction (when and what). Based on the past sequence $S$ of diagnostic trouble codes (DTCs), manufacturers want to prevent error patterns (EPs) from happening by predicting their likelihood and time of occurrence.
• Figure 2: Error Pattern Prediction using Multimodal Sequences. Past diagnostic trouble codes only provide limited information about complex and overlapping error patterns. In addition, domain experts rely on environmental conditions (e.g., temperature, voltage, humidity, …) to identify more accurately error patterns.
• Figure 3: Anonymized Causal Graph. Example of a sequence of events (DTCs) that lead to a steering wheel degradation and a power limitation as outcome labels. The causal indicators are shown in violet if inhibitory and orange or red if excitatory, depending on the magnitude. The voltage error DTC is the biggest causal driver for the power limitation EP while the security sync failure seems to have the biggest inhibitory effect on all EP.
• Figure 4: Anonymized Instance-Time Causal Graph for Diagnostic Defect Cascade. Temporal evolution of a diagnostic defect cascade in a vehicle ($|\mathcal{X}| \approx 29,100$). TRACE effectively captures causal relationships, revealing distinct error clusters at different time steps (e.g., initial sensor failures at $t=3$ triggering mechanical faults at $t=12$, battery at issue $t=17$). This enables actionable root-cause analysis by isolating the specific onset of a failure mechanism and its strength using the conditional mutual information.
• Figure 5: Time and Mileage Distribution. Distribution of $t_i$ and $m_i$ (relative time and mileage) in our dataset.

Theorems & Definitions (54)

• Definition 1: Counting Process
• Definition 2: Event Sequence
• Definition 3: Marked Event Sequence
• Definition 4: Conditional Intensity Function
• Remark 1
• Definition 5: Hawkes Process
• Definition 6: Mutually-Exciting Hawkes processes
• Remark 2
• Definition 7: Sentence as a Unit-Time Event Sequence
• Definition 8: Entropy