Autonomous Driving with a Deep Dual-Model Solution for Steering and Braking Control

TL;DR

This work addresses the resource constraints of autonomous driving systems by proposing a dual-model approach that uses PilotNet for steering and MobileNet SSD for braking, both driven by a single front-facing camera. A modified PilotNet employing depthwise separable convolutions and bottleneck layers reduces parameters, model size, and inference latency while maintaining comparable performance to the original. In GTA V-based simulations, the modified PilotNet achieves similar loss and MAE to the original but with substantially lower latency (≈167 ms) and memory footprint (≈3.66 MB vs 9.48 MB), validating the feasibility of running a modular steering/braking system on constrained hardware. The study demonstrates the advantages of specialization, parallel processing, and a simulation-based evaluation platform, and outlines avenues for future enhancements such as throttle prediction, obstacle detection expansion, and neural architecture search.

Abstract

The technology of autonomous driving is currently attracting a great deal of interest in both research and industry. In this paper, we present a deep learning dual-model solution that uses two deep neural networks for combined braking and steering in autonomous vehicles. Steering control is achieved by applying the NVIDIA's PilotNet model to predict the steering wheel angle, while braking control relies on the use of MobileNet SSD. Both models rely on a single front-facing camera for image input. The MobileNet SSD model is suitable for devices with constrained resources, whereas PilotNet struggles to operate efficiently on smaller devices with limited resources. To make it suitable for such devices, we modified the PilotNet model using our own original network design and reduced the number of model parameters and its memory footprint by approximately 60%. The inference latency has also been reduced, making the model more suitable to operate on resource-constrained devices. The modified PilotNet model achieves similar loss and accuracy compared to the original PilotNet model. When evaluated in a simulated environment, both autonomous driving systems, one using the modified PilotNet model and the other using the original PilotNet model for steering, show similar levels of autonomous driving performance.

Figures (2)

• Figure 1: Architecture of the proposed autonomous driving system
• Figure 2: Input images and feature maps of the second convolutional layer in PilotNet