Joint Optimization of DNN Model Caching and Request Routing in Mobile Edge Computing

TL;DR

This work tackles the problem of jointly caching dynamic DNN submodels and routing user requests in mobile edge computing to balance inference precision and model loading latency. It introduces CoCaR, an offline LP-based algorithm with randomized rounding, and CoCaR-OL, an online variant that leverages historical request patterns to adapt caching decisions in real time. By reformulating the problem with auxiliary variables A_{n,u,h} and providing theoretical guarantees, the methods achieve near-optimal performance, with CoCaR providing at least a 46% gain in average inference precision and CoCaR-OL delivering substantial QoE improvements in online settings. The results demonstrate that fine-grained dynamic DNN caching can significantly improve edge resource utilization, user QoE, and cache hit rates in MEC environments with limited cache capacity and volatile request patterns.

Abstract

Mobile edge computing (MEC) can pre-cache deep neural networks (DNNs) near end-users, providing low-latency services and improving users' quality of experience (QoE). However, caching all DNN models at edge servers with limited capacity is difficult, and the impact of model loading time on QoE remains underexplored. Hence, we introduce dynamic DNNs in edge scenarios, disassembling a complete DNN model into interrelated submodels for more fine-grained and flexible model caching and request routing solutions. This raises the pressing issue of jointly deciding request routing and submodel caching for dynamic DNNs to balance model inference precision and loading latency for QoE optimization. In this paper, we study the joint dynamic model caching and request routing problem in MEC networks, aiming to maximize user request inference precision under constraints of server resources, latency, and model loading time. To tackle this problem, we propose CoCaR, an offline algorithm based on linear programming and random rounding that leverages dynamic DNNs to optimize caching and routing schemes, achieving near-optimal performance. Furthermore, we develop an online variant of CoCaR, named CoCaR-OL, enabling effective adaptation to dynamic and unpredictable online request patterns. The simulation results demonstrate that the proposed CoCaR improves the average inference precision of user requests by 46\% compared to state-of-the-art baselines. In addition, in online scenarios, CoCaR-OL achieves an improvement of no less than 32.3\% in user QoE over competitive baselines.

Figures (14)

• Figure 1: Submodels division and switching of ViT. When switching from submodel 1 to submodel 2 of ViT, we only need to remove $\text{ExtNet}_1$ of submodel 1 and then connect $\text{HidNet}_2$ and $\text{ExtNet}_2$ to form submodel 2.
• Figure 2: Examples of static DNN and dynamic DNN schemes.
• Figure 3: System model. The same color indicates the dynamic DNN associated with a model type, while different lengths of that color represent different submodels of the dynamic DNN.
• Figure 4: Communication latency: Routing user $u$'s request involves wireless transmission from $u$ to home BS $n_1$, wired transmission from $n_1$ to target BS $n_3$, and a total of 6 hops from initiating the request to receive the inference result.
• Figure 5: Illustration of model caching in an online scenario. There are two types of models, $A$ and $B$, each comprising three distinct submodels. At time slot $t_1$, the system decides to cache submodel $A_2$, which requires downloading the additional component $\Delta A_2$ from the cloud to switch from $A_1$ to $A_2$. To satisfy the cache capacity constraint, submodel $B_3$ is simultaneously reduced to $B_2$. Since downloading $\Delta A_2$ takes two time slots, $A_2$ becomes available to serve users only from time slot $t_3$ onward. In contrast, model eviction is fast, allowing $B_2$ to serve users immediately at $t_2$.

Theorems & Definitions (12)

• Lemma 1
• proof
• Lemma 2
• proof
• Definition 1: Chernoff Bound chernoff
• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3