OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation

TL;DR

OneAdapt tackles the challenge of resource-efficient, accurate streaming media analytics by using a gradient-based knob adaptation method. It introduces AccGrad, approximated efficiently via OutputGrad, which decouples into InputGrad and DNNGrad to update configurations with a simple gradient ascent rule. The approach achieves frequent, near-optimal adaptation with low overhead, outperforming profiling- and heuristic-based baselines across multiple tasks, data types, and knobs. Empirical results show substantial bandwidth and GPU savings, along with modest or improved accuracy, highlighting practical impact for edge-to-cloud video analytics and similar pipelines. Limitations include discrete knob handling and applicability boundaries beyond streaming analytics, suggesting avenues for future optimization and broader validation.

Abstract

Deep learning inference on streaming media data, such as object detection in video or LiDAR feeds and text extraction from audio waves, is now ubiquitous. To achieve high inference accuracy, these applications typically require significant network bandwidth to gather high-fidelity data and extensive GPU resources to run deep neural networks (DNNs). While the high demand for network bandwidth and GPU resources could be substantially reduced by optimally adapting the configuration knobs, such as video resolution and frame rate, current adaptation techniques fail to meet three requirements simultaneously: adapt configurations (i) with minimum extra GPU or bandwidth overhead; (ii) to reach near-optimal decisions based on how the data affects the final DNN's accuracy, and (iii) do so for a range of configuration knobs. This paper presents OneAdapt, which meets these requirements by leveraging a gradient-ascent strategy to adapt configuration knobs. The key idea is to embrace DNNs' differentiability to quickly estimate the accuracy's gradient to each configuration knob, called AccGrad. Specifically, OneAdapt estimates AccGrad by multiplying two gradients: InputGrad (i.e. how each configuration knob affects the input to the DNN) and DNNGrad (i.e. how the DNN input affects the DNN inference output). We evaluate OneAdapt across five types of configurations, four analytic tasks, and five types of input data. Compared to state-of-the-art adaptation schemes, OneAdapt cuts bandwidth usage and GPU usage by 15-59% while maintaining comparable accuracy or improves accuracy by 1-5% while using equal or fewer resources.

Figures (14)

• Figure 1: Demonstrating the adaptive behavior of OneAdapt compared to alternatives: OneAdapt quickly adapts to a near-optimal configuration after the change of input content, whereas the profiling-based method adapts slowly and the heuristic-based method adapts quickly but suboptimally.
• Figure 2: In streaming media analytics, data from sensors is processed by the input filtering module before being sent to the DNN on constrained GPU resources. Given the DNN module might be on a remote device or cloud, data transmission can occur over a bandwidth-limited network.
• Figure 3: Illustrating two types of prior work: profiling and heuristics. The profiling-based approach can obtain optimal configuration but runs a lot of extra DNN inferences, while heuristics run no extra DNN inference but may pick suboptimal configuration (§\ref{['subsec:configuration-adaptation']}).
• Figure 4: Illustrating how OneAdapt estimates AccGrad using InputGrad and DNNGrad using the sensor and the server.
• Figure 5: Comparison between calculating OutputGrad naively and calculating OutputGrad using OneAdapt. OneAdapt calculates OutputGrad with no extra inference.