Enhancing Predictability of Multi-Tenant DNN Inference for Autonomous Vehicles' Perception

TL;DR

This work tackles the challenge of achieving temporal and functional predictability in autonomous-vehicle perception with multi-tenant DNNs. It introduces PP-DNN, a system that dynamically selects critical frames and ROIs, predicts FLOPs, and coordinates multi-task inference to bound latency while preserving accuracy. The architecture comprises an ROI generator, FLOPs predictor, ROI scheduler, and a detection predictor, all integrated into a ROS-based AV pipeline and evaluated on BDD100K and nuScenes. Results show large gains in fusion frame throughput, reduced fusion delay and its variability, improved detection completeness, and substantially better cost-effectiveness, demonstrating practical viability for real-time, predictable AV perception.

Abstract

Autonomous vehicles (AVs) rely on sensors and deep neural networks (DNNs) to perceive their surrounding environment and make maneuver decisions in real time. However, achieving real-time DNN inference in the AV's perception pipeline is challenging due to the large gap between the computation requirement and the AV's limited resources. Most, if not all, of existing studies focus on optimizing the DNN inference time to achieve faster perception by compressing the DNN model with pruning and quantization. In contrast, we present a Predictable Perception system with DNNs (PP-DNN) that reduce the amount of image data to be processed while maintaining the same level of accuracy for multi-tenant DNNs by dynamically selecting critical frames and regions of interest (ROIs). PP-DNN is based on our key insight that critical frames and ROIs for AVs vary with the AV's surrounding environment. However, it is challenging to identify and use critical frames and ROIs in multi-tenant DNNs for predictable inference. Given image-frame streams, PP-DNN leverages an ROI generator to identify critical frames and ROIs based on the similarities of consecutive frames and traffic scenarios. PP-DNN then leverages a FLOPs predictor to predict multiply-accumulate operations (MACs) from the dynamic critical frames and ROIs. The ROI scheduler coordinates the processing of critical frames and ROIs with multiple DNN models. Finally, we design a detection predictor for the perception of non-critical frames. We have implemented PP-DNN in an ROS-based AV pipeline and evaluated it with the BDD100K and the nuScenes dataset. PP-DNN is observed to significantly enhance perception predictability, increasing the number of fusion frames by up to 7.3x, reducing the fusion delay by >2.6x and fusion-delay variations by >2.3x, improving detection completeness by 75.4% and the cost-effectiveness by up to 98% over the baseline.

Figures (12)

• Figure 1: A general autonomous driving pipeline.
• Figure 2: The timing analysis of the detection completeness ratio for each frame when FPS = 5.
• Figure 3: An example of environment-aware dynamic ROIs.
• Figure 4: The area ratio for consecutive frames and the corresponding inference time using Faster R-CNN.
• Figure 5: GMACs of different image sizes (Height, Width) for DNN models in the perception.