Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative Learning-Based Framework for VNF Caching and Placement Optimization over Low Earth Orbit Satellite Networks

Khai Doan, Marios Avgeris, Aris Leivadeas, Ioannis Lambadaris, Wonjae Shin

TL;DR

This work tackles end-to-end service deployment in Low Earth Orbit satellite networks by optimizing SFC placement under partial VNF caching and dynamic topology. It combines a DP-based offline solution for optimal placement with a cooperative MAQL framework for online, decentralized decision making, and introduces a BO-based caching policy to select which VNFs to pre-install on satellites. The key contributions are (i) a parameter-sharing MAQL strategy that mitigates non-stationarity in a periodic ISL environment, (ii) a BO-driven mechanism to maximize the request serving rate via smart caching, and (iii) extensive evaluation showing near-optimal placement performance and enhanced serving rates compared with baselines. The results demonstrate practical gains for LEO mega-constellations, enabling more responsive, scalable SFC deployment with limited per-satellite resources.

Abstract

Low Earth Orbit Satellite Networks (LSNs) are integral to supporting a broad range of modern applications, which are typically modeled as Service Function Chains (SFCs). Each SFC is composed of Virtual Network Functions (VNFs), where each VNF performs a specific task. In this work, we tackle two key challenges in deploying SFCs across an LSN. Firstly, we aim to optimize the long-term system performance by minimizing the average end-to-end SFC execution delay, given that each satellite comes with a pre-installed/cached subset of VNFs. To achieve optimal SFC placement, we formulate an offline Dynamic Programming (DP) equation. To overcome the challenges associated with DP, such as its complexity, the need for probability knowledge, and centralized decision-making, we put forth an online Multi-Agent Q-Learning (MAQL) solution. Our MAQL approach addresses convergence issues in the non-stationary LSN environment by enabling satellites to share learning parameters and update their Q-tables based on distinct rules for their selected actions. Secondly, to determine the optimal VNF subsets for satellite caching, we develop a Bayesian Optimization (BO)-based learning mechanism that operates both offline and continuously in the background during runtime. Extensive experiments demonstrate that our MAQL approach achieves near-optimal performance comparable to the DP model and significantly outperforms existing baselines. Moreover, the BO-based approach effectively enhances the request serving rate over time.

Cooperative Learning-Based Framework for VNF Caching and Placement Optimization over Low Earth Orbit Satellite Networks

TL;DR

This work tackles end-to-end service deployment in Low Earth Orbit satellite networks by optimizing SFC placement under partial VNF caching and dynamic topology. It combines a DP-based offline solution for optimal placement with a cooperative MAQL framework for online, decentralized decision making, and introduces a BO-based caching policy to select which VNFs to pre-install on satellites. The key contributions are (i) a parameter-sharing MAQL strategy that mitigates non-stationarity in a periodic ISL environment, (ii) a BO-driven mechanism to maximize the request serving rate via smart caching, and (iii) extensive evaluation showing near-optimal placement performance and enhanced serving rates compared with baselines. The results demonstrate practical gains for LEO mega-constellations, enabling more responsive, scalable SFC deployment with limited per-satellite resources.

Abstract

Low Earth Orbit Satellite Networks (LSNs) are integral to supporting a broad range of modern applications, which are typically modeled as Service Function Chains (SFCs). Each SFC is composed of Virtual Network Functions (VNFs), where each VNF performs a specific task. In this work, we tackle two key challenges in deploying SFCs across an LSN. Firstly, we aim to optimize the long-term system performance by minimizing the average end-to-end SFC execution delay, given that each satellite comes with a pre-installed/cached subset of VNFs. To achieve optimal SFC placement, we formulate an offline Dynamic Programming (DP) equation. To overcome the challenges associated with DP, such as its complexity, the need for probability knowledge, and centralized decision-making, we put forth an online Multi-Agent Q-Learning (MAQL) solution. Our MAQL approach addresses convergence issues in the non-stationary LSN environment by enabling satellites to share learning parameters and update their Q-tables based on distinct rules for their selected actions. Secondly, to determine the optimal VNF subsets for satellite caching, we develop a Bayesian Optimization (BO)-based learning mechanism that operates both offline and continuously in the background during runtime. Extensive experiments demonstrate that our MAQL approach achieves near-optimal performance comparable to the DP model and significantly outperforms existing baselines. Moreover, the BO-based approach effectively enhances the request serving rate over time.
Paper Structure (27 sections, 26 equations, 11 figures, 3 tables, 3 algorithms)

This paper contains 27 sections, 26 equations, 11 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. VNF Placement in Static Satellite Topologies
  4. VNF Placement in Dynamic Satellite Topologies
  5. Novelty of the Proposed Framework
  6. System Model
  7. Communication Model
  8. SFCs, VNFs and Service Request Arrivals
  9. VNF Execution and Inter-Satellite Data Transfer
  10. VNF Placement in LSNs
  11. Dynamic Programming Formulation
  12. Multi-Agent Q-Learning Solution
  13. Agent's State
  14. Agent's Action
  15. Instant Cost
  16. ...and 12 more sections

Figures (11)

  • Figure 1: Evolution of $(v,u)$-ISL in time, when $e_{v,u}\left(1\right)=1, \tau_{v,u}=2$, $T_{v,u}=3$. Two satellites signify an active ISL.
  • Figure 2: The VNF/service placement problem on an LSN.
  • Figure 3: Visualization of SFC deployment in Example 2.
  • Figure 4: Visualization of the request transfer and satellites' buffered requests in Example 3.
  • Figure 5: Evaluation of the proposed MAQL-based scheme in comparison with the optimal DP solution.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Example 1
  • Example 2
  • Example 3