Collaborative Inference in DNN-based Satellite Systems with Dynamic Task Streams

TL;DR

The paper tackles latency-constrained, dynamic DNN inference for satellite constellations by marrying multi-exit DNNs with model partitioning across HEO and LEO nodes. It formulates a joint EE/partition optimization to maximize a task gain that blends completion rate and inference accuracy, and proves the problem is NP-hard by reducing it to a group knapsack. A task gain-aware offline DP-based algorithm is proposed to select EE and partition points, and simulations using CIFAR-10 and AlexNet demonstrate superior performance over baselines under realistic inter-satellite links. The work advances practical in-orbit AI by enabling adaptive, collaborative inference that respects stringent latency while leveraging inter-satellite resources, with potential impact on real-time Earth observation tasks. $G_n$ combines discrete accuracy rewards with a smooth satisfaction term, enabling nuanced latency-accuracy tradeoffs in dynamic task streams.$

Abstract

As a driving force in the advancement of intelligent in-orbit applications, DNN models have been gradually integrated into satellites, producing daily latency-constraint and computation-intensive tasks. However, the substantial computation capability of DNN models, coupled with the instability of the satellite-ground link, pose significant challenges, hindering timely completion of tasks. It becomes necessary to adapt to task stream changes when dealing with tasks requiring latency guarantees, such as dynamic observation tasks on the satellites. To this end, we consider a system model for a collaborative inference system with latency constraints, leveraging the multi-exit and model partition technology. To address this, we propose an algorithm, which is tailored to effectively address the trade-off between task completion and maintaining satisfactory task accuracy by dynamically choosing early-exit and partition points. Simulation evaluations show that our proposed algorithm significantly outperforms baseline algorithms across the task stream with strict latency constraints.

Figures (6)

• Figure 1: The workflow of collaborative task stream offloading system.
• Figure 2: AlexNet model consists of 4 EE points. When we consider task failure as another particular case when starting, AlexNet model has 5 EE points.
• Figure 3: The total gain in different the number of tasks.
• Figure 4: The total gain under the different task arrive rate $p$.
• Figure 5: The task completion rate in different task numbers.