DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle

TL;DR

The paper addresses autonomous driving reliability under poor illumination by replacing camera-based perception with LiDAR-centric processing. It introduces DeepIPCv2, which uses PolarNet to segment LiDAR point clouds and projects them into front-view and BEV representations, fused into a latent that a GRU-based controller decodes through command-specific MLPs and PID controls to produce steering, throttle, and waypoints. Ablation studies show that segmentation-focused, multi-view LiDAR perception yields superior drivability across noon, evening, and night, outperforming camera-LiDAR fusion baselines in low-light scenarios. The results demonstrate robust, end-to-end driving performance with reduced need for interventions, highlighting the practical viability of LiDAR-driven perception in challenging lighting conditions.

Abstract

We present DeepIPCv2, an autonomous driving model that perceives the environment using a LiDAR sensor for more robust drivability, especially when driving under poor illumination conditions where everything is not clearly visible. DeepIPCv2 takes a set of LiDAR point clouds as the main perception input. Since point clouds are not affected by illumination changes, they can provide a clear observation of the surroundings no matter what the condition is. This results in a better scene understanding and stable features provided by the perception module to support the controller module in estimating navigational control properly. To evaluate its performance, we conduct several tests by deploying the model to predict a set of driving records and perform real automated driving under three different conditions. We also conduct ablation and comparative studies with some recent models to justify its performance. Based on the experimental results, DeepIPCv2 shows a robust performance by achieving the best drivability in all driving scenarios. Furthermore, to support future research, we will upload the codes and data to https://github.com/oskarnatan/DeepIPCv2.

Figures (6)

• Figure 1: DeepIPCv2 perceives the environment by encoding a set of segmented point clouds that are projected into front and top-view perspectives. Then, the extracted features are used to estimate waypoints (white dots) and navigational controls to drive the vehicle following the route points (white hollow circles). Meanwhile, the RGBD image is only for record purposes. It shows how the RGBD camera fails in capturing surrounding information as it cannot provide a clearly visible set of RGB image and depth map. Therefore, DeepIPCv2 employs a LiDAR sensor which has its own lasers as the light source to provide point clouds that are not affected by poor illumination conditions. The detailed architecture of DeepIPCv2 can be seen in Fig. \ref{['fig:model']}.
• Figure 2: The architecture of DeepIPCv2. The blue and green blocks are the perception and controller modules respectively. Darker blocks are trainable, while light-colored blocks are not. In the perception module, PolarNet polarnet is employed to support point cloud segmentation. Then, the architecture of encoders and feature fusion modules can be seen in Fig. \ref{['fig:model2']}.
• Figure 3: The architecture of the encoders and the feature fusion module. We use atrous convolution blocks atrous_conv with different kernel sizes and dilation rates to capture low-level features from the projected point clouds that have vacant regions. Then, both top and front features are fused together and their relationship are learned by the feature fusion module.
• Figure 4: The experiment areas (more detail at https://goo.gl/maps/9rXobdhP3VYdjXn48). White hollow circles represent a route that consists of a start, finish, and route points.
• Figure 5: Sensor placement on the robotic vehicle. The vehicle is categorized as a nonholonomic unicycle since it has motorized rear wheels and omnidirectional front wheels.