Imitation Learning for Autonomous Driving: Insights from Real-World Testing

TL;DR

The paper tackles real-time autonomous steering by casting it as imitation learning and evaluating end-to-end CNN-based architectures on the MIT Racecar. An incremental design approach is used, starting from a PD baseline and advancing to CNN, CNN-LSTM, and CNN-NODE, with real-time inference via TensorRT. Key findings show the PD system struggles with sharp turns and lighting changes, CNN improves steering but lacks temporal context, CNN-LSTM provides smoother and more robust driving, and CNN-NODE with RK4 offers comparable performance to CNN-LSTM while modeling continuous dynamics. The work highlights the importance of diverse real-world data, multi-camera viewpoints, and solver choices for NODEs, contributing practical insights for deploying robust, real-time imitation-learning controllers in autonomous driving.

Abstract

This work focuses on the design of a deep learning-based autonomous driving system deployed and tested on the real-world MIT Racecar to assess its effectiveness in driving scenarios. The Deep Neural Network (DNN) translates raw image inputs into real-time steering commands in an end-to-end learning fashion, following the imitation learning framework. The key design challenge is to ensure that DNN predictions are accurate and fast enough, at a high sampling frequency, and result in smooth vehicle operation under different operating conditions. In this study, we design and compare various DNNs, to identify the most effective approach for real-time autonomous driving. In designing the DNNs, we adopted an incremental design approach that involved enhancing the model capacity and dataset to address the challenges of real-world driving scenarios. We designed a PD system, CNN, CNN-LSTM, and CNN-NODE, and evaluated their performance on the real-world MIT Racecar. While the PD system handled basic lane following, it struggled with sharp turns and lighting variations. The CNN improved steering but lacked temporal awareness, which the CNN-LSTM addressed as it resulted in smooth driving performance. The CNN-NODE performed similarly to the CNN-LSTM in handling driving dynamics, yet with slightly better driving performance. The findings of this research highlight the importance of iterative design processes in developing robust DNNs for autonomous driving applications. The experimental video is available at https://www.youtube.com/watch?v=FNNYgU--iaY.

Figures (9)

• Figure 1: Test environment.
• Figure 2: Error representation on an example frame.
• Figure 3: Inference structure of the CNN model.
• Figure 4: Testing performance of the CNN
• Figure 5: Inference structure of the CNN-LSTM and CNN-NODE.