Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

TL;DR

The paper tackles autonomous vehicle steering by exploiting cooperative perception through V2V image sharing. It introduces a CNN-LSTM-FC end-to-end network that processes paired image sequences from the host vehicle and a ahead vehicle to predict steering, demonstrating state-of-the-art performance on the Udacity dataset. Key contributions include the first end-to-end cooperative steering system, the use of longer image sequences, and comprehensive comparisons against strong single-vehicle baselines. The study validates that sharing forward-looking visual information enhances steering accuracy and robustness, with plans to test on real V2V data in future work.

Abstract

A fundamental challenge in autonomous vehicles is adjusting the steering angle at different road conditions. Recent state-of-the-art solutions addressing this challenge include deep learning techniques as they provide end-to-end solution to predict steering angles directly from the raw input images with higher accuracy. Most of these works ignore the temporal dependencies between the image frames. In this paper, we tackle the problem of utilizing multiple sets of images shared between two autonomous vehicles to improve the accuracy of controlling the steering angle by considering the temporal dependencies between the image frames. This problem has not been studied in the literature widely. We present and study a new deep architecture to predict the steering angle automatically by using Long-Short-Term-Memory (LSTM) in our deep architecture. Our deep architecture is an end-to-end network that utilizes CNN, LSTM and fully connected (FC) layers and it uses both present and futures images (shared by a vehicle ahead via Vehicle-to-Vehicle (V2V) communication) as input to control the steering angle. Our model demonstrates the lowest error when compared to the other existing approaches in the literature.

Figures (9)

• Figure 1: The overview of our proposed vehicle-assisted end-to-end system. Vehicle 2 (V2) sends his information to Vehicle 1 (V1) over V2V communication. V1 combines that information along with its own information to control the steering angle. The prediction is made through our CNN+LSTM+FC network (see Fig. \ref{['fig:model']} for the details of our network).
• Figure 2: CNN + LSTM + FC Image sharing model. Our model uses 5 convolutional layers, followed by 3 LSTM layers, followed by 4 FC layers. See Table \ref{['table:proposed_arq']} for further details of our proposed architecture.
• Figure 3: The angle distribution within the entire Udacity dataset (angle in radians vs. total number of frames), just angles between -1 and 1 radians are shown.
• Figure 4: An overview of the used baseline models in this paper. The details of each model can be found in their respective source paper.
• Figure 5: Training and Validation steps for our best model with $x=8$.