Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained MARL for Coexisting TN-NTN Resource Allocation: Scalability and Flexibility

Cuong Le, Thang X. Vu, Stefano Andrenacci, Symeon Chatzinotas

TL;DR

This work tackles scalable downlink resource allocation in coexisting terrestrial and non-terrestrial networks under QoS constraints and dynamic user behavior. It introduces a decomposition-based constrained MARL framework that splits subchannels into subset resource blocks (SRBs) and trains multiple agents per base station, using MAPPO with Lagrangian relaxation to enforce QoS. A dynamic training environment augments policy robustness to varying numbers and locations of active users. Large-scale simulations on a 20 MHz band with up to 320 subchannels demonstrate improved scalability and robustness over baseline MARL methods, highlighting practical relevance for 6G-like TN-NTN deployments.

Abstract

This paper considers the joint TN-NTN constrained resource allocation, where terrestrial base stations and non-terrestrial base stations coexist in the spectrum. We focus on large-scale and practical scenarios characterized by large numbers of transmission channels and users, alongside highly dynamic user behaviors. As common learning solutions fail to address these challenges, we propose a decomposition solution based on the special properties of the cross-segment interference, and then tackle the original problem via solving subproblems in a sequential learning manner. Furthermore, to enhance the flexibility of the learned policies, we design a stochastic training environment that captures the key characteristics of real-world systems. Simulation results tested on the full 20MHz bandwidth with various numerologies show that our solution significantly improves scalability compared to existing solutions and remains robust in highly dynamic scenarios.

Constrained MARL for Coexisting TN-NTN Resource Allocation: Scalability and Flexibility

TL;DR

This work tackles scalable downlink resource allocation in coexisting terrestrial and non-terrestrial networks under QoS constraints and dynamic user behavior. It introduces a decomposition-based constrained MARL framework that splits subchannels into subset resource blocks (SRBs) and trains multiple agents per base station, using MAPPO with Lagrangian relaxation to enforce QoS. A dynamic training environment augments policy robustness to varying numbers and locations of active users. Large-scale simulations on a 20 MHz band with up to 320 subchannels demonstrate improved scalability and robustness over baseline MARL methods, highlighting practical relevance for 6G-like TN-NTN deployments.

Abstract

This paper considers the joint TN-NTN constrained resource allocation, where terrestrial base stations and non-terrestrial base stations coexist in the spectrum. We focus on large-scale and practical scenarios characterized by large numbers of transmission channels and users, alongside highly dynamic user behaviors. As common learning solutions fail to address these challenges, we propose a decomposition solution based on the special properties of the cross-segment interference, and then tackle the original problem via solving subproblems in a sequential learning manner. Furthermore, to enhance the flexibility of the learned policies, we design a stochastic training environment that captures the key characteristics of real-world systems. Simulation results tested on the full 20MHz bandwidth with various numerologies show that our solution significantly improves scalability compared to existing solutions and remains robust in highly dynamic scenarios.
Paper Structure (17 sections, 4 theorems, 17 equations, 4 figures)

This paper contains 17 sections, 4 theorems, 17 equations, 4 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Conventional optimization formulation
  4. Standard Dec-POMDP Reformulation
  5. Proposed Framework
  6. Proposed Formulation with Decomposition
  7. Relation with the Original Formulation
  8. Details of the Proposed Algorithm
  9. Reward function
  10. Action space
  11. Observation
  12. State space
  13. Cost function
  14. Constraint Handling
  15. Enhance Flexibility via Dynamic Environment Training
  16. ...and 2 more sections

Key Result

Lemma 1

Let $\Bar{J}_m(\Bar{\pi}_m|\Bar{\pi}_{1:m-1})$ be the objective value of prob:cmdp_decomposed at the policy $\Bar{\pi}_m$ w.r.t. $\Bar{\pi}_{1:m-1}$, and $J({\Bar{\pi}})$ be the objective value of prob:cmdp at the joint policy $\Bar{\pi}=\prod_{m=1}^M\Bar{\pi}_m$. Then $J(\Bar{\pi}) =\sum_{m=1}^M\Ba

Figures (4)

  • Figure 1: Illustration of the decomposed problem with BSs and three SRBs, with each SRB at a BS controlled by one agent.
  • Figure 2: The computational model of the proposed solution, where the action of each BS in each time step is calculated sequentially by its $M$ agents.
  • Figure 3: The scalability of all solutions demonstrated by the learning performance on different number of users and subchannels.
  • Figure 4: The flexibility and robustness of learned policies demonstrated by testing results on a real-world scenario.

Theorems & Definitions (5)

  • Lemma 1
  • Lemma 2
  • Theorem 1: Optimality preservation
  • Claim 1
  • Proposition 1