TVR: Automotive System Requirement Traceability Validation and Recovery Through Retrieval-Augmented Generation

TL;DR

This work tackles the challenge of validating and recovering traceability between automotive stakeholder and system requirements, with a focus on Diagnostic Trouble Code (DTC) management. It introduces TVR, a retrieval-augmented generation framework that uses a retriever to provide in-context demonstrations and a generator to produce Yes/No validations, complemented by a three-step traceability-recovery workflow. Across an industrial dataset of 2,132 links, TVR achieves up to 98.87% validation accuracy and 85.50% recovery correctness, markedly outperforming retrieval-only and other LLM baselines, and demonstrates robustness to unseen requirement variations. The approach offers practical, scalable automation for automotive traceability and has potential applicability to other domains requiring stakeholder-to-system requirement linkage with domain-specific terminology and safety-critical constraints.

Abstract

In automotive software development, as well as other domains, traceability between stakeholder requirements and system requirements is crucial to ensure consistency, correctness, and regulatory compliance. However, erroneous or missing traceability relationships often arise due to improper propagation of requirement changes or human errors in requirement mapping, leading to inconsistencies and increased maintenance costs. Existing approaches do not address traceability between stakeholder and system requirements, and are not validated on industrial data, where the links between requirements are established manually by engineers. Additionally, automotive requirements often exhibit variations in the way they are expressed, posing challenges for training-based approaches. Recent advancements in large language models (LLMs) provide new opportunities to address these challenges. In this paper, we introduce TVR, a requirement Traceability Validation and Recovery approach primarily targeting automotive systems, leveraging LLMs enhanced with retrieval-augmented generation (RAG). TVR is designed to validate existing traceability links and recover missing ones with high accuracy. The experimental results highlight the practical effectiveness of TVR in industrial settings, offering a promising solution for improving requirements traceability in complex automotive systems.

Figures (5)

• Figure 1: Fictitious Variations of Stakeholder Requirements (Due to data privacy concerns, only fictitious examples are shown here. Italicized variables in the examples (e.g., MESSAGE_1) are pseudonyms—real values were employed in the experiments.).
• Figure 2: Example of Sanitized DTC System Requirement. Note: Real values were employed in the experiments.
• Figure 3: Overall Framework of TVR.
• Figure 4: Input Prompt.
• Figure 5: F1 Score Comparison for Euclidean and Cosine Similarity Across Different Values of K.