Commonsense Reasoning-Aided Autonomous Vehicle Systems

TL;DR

This paper addresses the gap where deep learning excels at perception but struggles with high-level road reasoning. It proposes a hybrid autonomous vehicle framework that attaches a dedicated commonsense reasoning layer, built on image-derived facts and domain-expert rules, to guide safe and explainable decisions. The approach leverages a Prolog-based knowledge base to model collective vehicle behaviors and enforce safety constraints, functioning as a separate, updatable layer alongside perception. Preliminary CARLA-based results indicate that the commonsense module improves accuracy in traffic-light interpretation and obstacle detection when combined with deep learning, offering potential improvements in safety, explainability, and regulatory alignment; future work will explore ASP-based enhancements, real-world datasets, and broader domain applications.

Abstract

Autonomous Vehicle (AV) systems have been developed with a strong reliance on machine learning techniques. While machine learning approaches, such as deep learning, are extremely effective at tasks that involve observation and classification, they struggle when it comes to performing higher level reasoning about situations on the road. This research involves incorporating commonsense reasoning models that use image data to improve AV systems. This will allow AV systems to perform more accurate reasoning while also making them more adjustable, explainable, and ethical. This paper will discuss the findings so far and motivate its direction going forward.