Transforming Vehicle Diagnostics: A Multimodal Approach to Error Patterns Prediction

TL;DR

The paper tackles the limitation of using only Diagnostic Trouble Codes (DTCs) for vehicle fault diagnosis by incorporating rich environmental context. It introduces BiCarFormer, a multimodal bidirectional Transformer that fuses DTC embeddings and environmental-condition embeddings via co-attention to predict error patterns (EPs). On a large real-world automotive dataset with 360 EPs, BiCarFormer outperforms sequence-only baselines (e.g., DTC-TranGRU, BERT) in AUROC and F1, with notable gains on rare classes and per-instance predictions, while providing interpretability through cross-attention analyses. The work highlights practical benefits for diagnostic automation and outlines downstream opportunities in unsupervised anomaly detection and explainability within domain-knowledge-driven pipelines.

Abstract

Accurately diagnosing and predicting vehicle malfunctions is crucial for maintenance and safety in the automotive industry. While modern diagnostic systems primarily rely on sequences of vehicular Diagnostic Trouble Codes (DTCs) registered in On-Board Diagnostic (OBD) systems, they often overlook valuable contextual information such as raw sensory data (e.g., temperature, humidity, and pressure). This contextual data, crucial for domain experts to classify vehicle failures, introduces unique challenges due to its complexity and the noisy nature of real-world data. This paper presents BiCarFormer: the first multimodal approach to multi-label sequence classification of error codes into error patterns that integrates DTC sequences and environmental conditions. BiCarFormer is a bidirectional Transformer model tailored for vehicle event sequences, employing embedding fusions and a co-attention mechanism to capture the relationships between diagnostic codes and environmental data. Experimental results on a challenging real-world automotive dataset with 22,137 error codes and 360 error patterns demonstrate that our approach significantly improves classification performance compared to models that rely solely on DTC sequences and traditional sequence models. This work highlights the importance of incorporating contextual environmental information for more accurate and robust vehicle diagnostics, hence reducing maintenance costs and enhancing automation processes in the automotive industry.

Figures (8)

• Figure 1: Error Pattern (EP) prediction based on past Diagnostic Trouble Codes (DTCs) and environmental conditions (e.g., temperature, voltage, …).
• Figure 2: Temporal and spatial point process representation of events from a vehicle. Bold vertical lines indicate multiple events happening at the same time $t_i$ or mileage $m_i$.
• Figure 3: BiCarFormer overall architecture with multimodal masking. Both parallel transformers are computing cross attention scores conditioned on each modality $\boldsymbol{Q, K, V}$. Two final representations are generated for each modality: DTC ($\boldsymbol{H}_d)$ and e. conditions ($\boldsymbol{H}_e$). Multiple embeddings are defined at the input level to take into account token-specific features.
• Figure 4: Pretraining Base-DTC classification loss comparison with and without multimodal learning.
• Figure 5: Amount of attention received by each e. condition from the DTCs. The y-axis was truncated to improve clarity as well as the number of heads printed. We take $\boldsymbol{A}_{dtc \rightarrow env}$ of the last layer.