Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Foundation CAN LM: A Pretrained Language Model For Automotive CAN Data

Akiharu Esashi, Pawissanutt Lertpongrujikorn, Justin Makino, Yuibi Fujimoto, Mohsen Amini Salehi

TL;DR

The paper addresses fragmentation in CAN-based automotive AI by proposing a foundation model that learns general representations from decoded CAN signals. It introduces a unified tokenization scheme for mixed discrete-continuous CAN signals and trains a BERT-like encoder with MLM on unlabeled data (~10k vehicles, 9 days used for training, 44 features, 1 Hz, 10-second windows). The pretrained backbone is then fine-tuned on two heterogeneous downstream tasks—binary collision detection and multi-class point-of-impact location—demonstrating cross-task generalization and competitive performance against task-specific baselines. Ablation and limitations reveal that capacity gains alone do not fix rare-event discrimination and highlight the benefits of longer temporal context and more diverse corpora for robust automotive intelligence.

Abstract

The Controller Area Network (CAN) bus provides a rich source of vehicular signals increasingly leveraged for applications in automotive and auto insurance domains, including collision detection, predictive maintenance, and driver risk modeling. Despite this potential, existing pipelines largely train isolated task-specific models on raw CAN data, with only limited efforts exploring decoded signals. Such fragmentation prevents shared representation learning and limits cross-task generalization. By contrast, natural language processing (NLP) and computer vision (CV) have been transformed by the foundation model paradigm: large-scale pretraining followed by task-specific adaptation. In this work, we introduce the foundation CAN model that demonstrates multi-objective downstream generalization using a single pretrained backbone. Our approach treats CAN data as a language: we pretrain on large-scale, unlabeled decoded CAN signals and fine-tune across heterogeneous auto insurance tasks. To enable this, we propose a unified tokenization scheme for mixed discrete-continuous signals and address challenges of temporal complexity and trip-specific variability. Our results show that one pretrained CAN model can adapt effectively to diverse predictive tasks, validating that the foundation modeling paradigm, proven in NLP and CV, also holds for CAN data. This establishes a new direction for generalizable representation learning in automotive AI.

Foundation CAN LM: A Pretrained Language Model For Automotive CAN Data

TL;DR

The paper addresses fragmentation in CAN-based automotive AI by proposing a foundation model that learns general representations from decoded CAN signals. It introduces a unified tokenization scheme for mixed discrete-continuous CAN signals and trains a BERT-like encoder with MLM on unlabeled data (~10k vehicles, 9 days used for training, 44 features, 1 Hz, 10-second windows). The pretrained backbone is then fine-tuned on two heterogeneous downstream tasks—binary collision detection and multi-class point-of-impact location—demonstrating cross-task generalization and competitive performance against task-specific baselines. Ablation and limitations reveal that capacity gains alone do not fix rare-event discrimination and highlight the benefits of longer temporal context and more diverse corpora for robust automotive intelligence.

Abstract

The Controller Area Network (CAN) bus provides a rich source of vehicular signals increasingly leveraged for applications in automotive and auto insurance domains, including collision detection, predictive maintenance, and driver risk modeling. Despite this potential, existing pipelines largely train isolated task-specific models on raw CAN data, with only limited efforts exploring decoded signals. Such fragmentation prevents shared representation learning and limits cross-task generalization. By contrast, natural language processing (NLP) and computer vision (CV) have been transformed by the foundation model paradigm: large-scale pretraining followed by task-specific adaptation. In this work, we introduce the foundation CAN model that demonstrates multi-objective downstream generalization using a single pretrained backbone. Our approach treats CAN data as a language: we pretrain on large-scale, unlabeled decoded CAN signals and fine-tune across heterogeneous auto insurance tasks. To enable this, we propose a unified tokenization scheme for mixed discrete-continuous signals and address challenges of temporal complexity and trip-specific variability. Our results show that one pretrained CAN model can adapt effectively to diverse predictive tasks, validating that the foundation modeling paradigm, proven in NLP and CV, also holds for CAN data. This establishes a new direction for generalizable representation learning in automotive AI.
Paper Structure (13 sections, 5 figures, 2 tables)

This paper contains 13 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Dataset and Preprocessing
  5. Unified Tokenization of Mixed Signals
  6. Foundation Model Pretraining
  7. Performance Evaluation
  8. Experimental Setup
  9. Downstream Tasks
  10. Experimental Results and Analysis
  11. Ablation Study
  12. Limitation and Discussion
  13. Conclusion and Future Work

Figures (5)

  • Figure 1: Traditional task-specific pipelines (top) require isolated preprocessing and models for each task. Our foundation model approach (bottom) uses unified tokenization and a single pretrained backbone that adapts to multiple downstream tasks through task-specific fine-tuning.
  • Figure 1: Evaluation results for binary classification across imbalance ratios.
  • Figure 2: Unified tokenization pipeline for mixed-mode CAN signals. Decoded CAN data is split into discrete variables (symbolic and enumerated) and continuous variables, processed through type-specific strategies, then merged into a unified token vocabulary for pretraining.
  • Figure 3: Foundation CAN Model (10:1 Ratio)
  • Figure 4: Baseline GLM (10:1 Ratio)