Towards Safety-Compliant Transformer Architectures for Automotive Systems
Sven Kirchner, Nils Purschke, Chengdong Wu, Alois Knoll
TL;DR
The paper addresses safety-certification challenges for transformer-based multimodal perception in autonomous driving by proposing a modular architecture with multiple modality-specific encoders that map inputs into a shared latent space $\mathcal{Z}$, followed by modality-agnostic decoders. This latent-space fusion, implemented via attention, enables intrinsic redundancy and informational enrichment, aligning with ISO 26262 ASIL decomposition through independent signal paths $E_i: \mathcal{X}_i \rightarrow \mathcal{Z}$ and decoders $D_j: \mathcal{Z} \rightarrow \mathcal{Y}_j$. Key contributions include formalizing the representational redundancy framework, detailing a concrete LiDAR-camera fusion workflow and cross-modal processing, and outlining steps toward verifiable fail-operational AI in automotive contexts. The approach aims to bridge safety engineering and modern AI by embedding safety principles into multimodal Transformer architectures, paving the way for certifiable AI systems in ADAS and autonomous driving.
Abstract
Transformer-based architectures have shown remarkable performance in vision and language tasks but pose unique challenges for safety-critical applications. This paper presents a conceptual framework for integrating Transformers into automotive systems from a safety perspective. We outline how multimodal Foundation Models can leverage sensor diversity and redundancy to improve fault tolerance and robustness. Our proposed architecture combines multiple independent modality-specific encoders that fuse their representations into a shared latent space, supporting fail-operational behavior if one modality degrades. We demonstrate how different input modalities could be fused in order to maintain consistent scene understanding. By structurally embedding redundancy and diversity at the representational level, this approach bridges the gap between modern deep learning and established functional safety practices, paving the way for certifiable AI systems in autonomous driving.