DyRoNet: Dynamic Routing and Low-Rank Adapters for Autonomous Driving Streaming Perception

TL;DR

This paper addresses the challenge of achieving low-latency yet high-accuracy streaming perception in autonomous driving by introducing DyRoNet, a dynamic routing framework that selects among a bank of environment-tuned streaming perception branches. A speed router uses frame-difference signals to infer environmental velocity and route inputs to the most suitable branch, while Low-Rank Adaptation (LoRA) enables efficient sub-model adaptation without full fine-tuning. The training objective combines a streaming perception loss with an effective-and-efficient loss, guiding both branch accuracy and router efficiency; experiments on Argoverse-HD (and NuScenes-H) show superior sAP at competitive inference times compared to state-of-the-art methods. The results establish a new benchmark for streaming perception and provide practical insights into dynamic model selection, router design, and LoRA-based adaptation for real-time autonomous driving systems.

Abstract

The advancement of autonomous driving systems hinges on the ability to achieve low-latency and high-accuracy perception. To address this critical need, this paper introduces Dynamic Routing Network (DyRoNet), a low-rank enhanced dynamic routing framework designed for streaming perception in autonomous driving systems. DyRoNet integrates a suite of pre-trained branch networks, each meticulously fine-tuned to function under distinct environmental conditions. At its core, the framework offers a speed router module, developed to assess and route input data to the most suitable branch for processing. This approach not only addresses the inherent limitations of conventional models in adapting to diverse driving conditions but also ensures the balance between performance and efficiency. Extensive experimental evaluations demonstrate the adaptability of DyRoNet to diverse branch selection strategies, resulting in significant performance enhancements across different scenarios. This work establishes a new benchmark for streaming perception and provides valuable engineering insights for future work.

Figures (8)

• Figure 1: Illustration of DyRoNet's adaptive selection mechanism in streaming perception, contrasting with static traditional methods in complex environments [Best viewed in color and enlarged].
• Figure 2: The DyRoNet Framework: This figure shows DyRoNet's architecture with a multi-branch network. Two branches are illustrated, each as a streaming perception sub-network. The upper right details the core architecture. Each branch processes the current frame $I_t$ and historical frames $I_{t-1}, I_{t-2}, \cdots, I_{t-n}$. Features are extracted by the backbone and neck, split into streams for current and historical frames, fused, then passed to the prediction head. The Speed Router selects the branch based on frame difference $\Delta I_t$ from $I_t$ and $I_{t-1}$.
• Figure 3: The mean curves of frame differences are depicted here. The four curves correspond to frame sizes of the original frame, 200$\times$200, 100$\times$100, and 50$\times$50. Notably, these curves show distinct fluctuations across different vehicle motion cases.
• Figure 4: Illustrative Examples of Varied Weather Conditions and Times of Day: (a) Sunny during Daytime, (b) Cloudy during Daytime, (c) Rainy during Daytime, (d) Rainy during Nighttime, (e) Sunny during Nighttime.
• Figure 5: Rapid Fluctuations in Performance Under Identical Weather Conditions: (a) Clip id: 00c561 shows a Streaming Average Precision (sAP) of 16.2% using the StreamYOLO-s model, (b) Clip id: 395560 demonstrates a significantly higher sAP of 48.3% under the same model and weather condition, illustrating the variability in model performance even under consistent environmental factors.