Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey

TL;DR

This survey addresses the challenge of integrating domain knowledge and traffic regulations into autonomous vehicle trajectory prediction and planning, identifying a gap in comprehensive reviews that unify symbolic and data-driven approaches. It categorizes knowledge integration methods into knowledge-graph/ontology, reinforcement learning, LLM/RAG, formal logic, hybrid symbolic-neural, and diffusion-based approaches, detailing representative techniques and their advantages and limitations. It highlights the shift toward foundation models and diffusion paradigms as a means to encode regulatory and spatial constraints while preserving adaptability, interpretability, and safety. The work provides a roadmap for building knowledge-aware AV systems that combine explicit rule representations with modern learning, aiming to improve robustness, safety guarantees, and real-world deployability.

Abstract

This comprehensive survey examines the integration of knowledge-based approaches in autonomous driving systems, specifically focusing on trajectory prediction and planning. We extensively analyze various methodologies for incorporating domain knowledge, traffic rules, and commonsense reasoning into autonomous driving systems. The survey categorizes and analyzes approaches based on their knowledge representation and integration methods, ranging from purely symbolic to hybrid neuro-symbolic architectures. We examine recent developments in logic programming, foundation models for knowledge representation, reinforcement learning frameworks, and other emerging technologies incorporating domain knowledge. This work systematically reviews recent approaches, identifying key challenges, opportunities, and future research directions in knowledge-enhanced autonomous driving systems. Our analysis reveals emerging trends in the field, including the increasing importance of interpretable AI, the role of formal verification in safety-critical systems, and the potential of hybrid approaches that combine traditional knowledge representation with modern machine learning techniques.