Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach

TL;DR

This paper tackles indoor wireless networks by integrating multiple APs, STAR-RIS units, and NOMA, formulating a MINLP that jointly optimizes UE association, NOMA clustering, decoding order, active beamforming, and STAR-RIS coefficients. It introduces a decomposition-based solution with two-stage matching and correlation-based K-means for association and pairing, and a convex approximation imitated deep reinforcement learning framework (CAMAPPO) to accelerate learning of both active and passive beamforming. CAMAPPO combines suboptimal solutions from successive convex approximation with a MAPPO-based multi-agent RL paradigm, achieving faster convergence and about 15% higher network utility than baselines in simulations. The results demonstrate the practical potential of CAMAPPO for real-time indoor optimization, enabling efficient and scalable control of multi-STAR-RIS aided NOMA networks. The work advances indoor RIS-NOMA design by providing a structured optimization pipeline and a learning-driven, fast-converging control scheme suitable for dynamic environments.

Abstract

Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) provides 360-degree full-space coverage, optimizing both transmission and reflection for improved network performance and dynamic control of the indoor environment. However, deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. In this work, a novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication. To address these, we formulate an optimization problem involving user assignment, access point (AP) beamforming, and STAR-RIS phase control. A decomposition approach is used to solve the complex problem efficiently, employing a many-to-one matching algorithm for user-AP assignment and K-means clustering for resource management. Additionally, multi-agent deep reinforcement learning (MADRL) is leveraged to optimize the control of the STAR-RIS. Within the proposed MADRL framework, a novel approach is introduced in which each decision variable acts as an independent agent, enabling collaborative learning and decision making. The MADRL framework is enhanced by incorporating convex approximation (CA), which accelerates policy learning through suboptimal solutions from successive convex approximation (SCA), leading to faster adaptation and convergence. Simulations demonstrate significant improvements in network utility compared to baseline approaches.

Figures (10)

• Figure 1: STAR-RISs architecture.
• Figure 2: NOMA aided Multi-STAR-RIS indoor networks.
• Figure 3: The proposed decomposition and solution framework for each subproblem and associated algorithm.
• Figure 4: Proposed Convex Approximation-imitated MAPPO.
• Figure 5: Analysis of learning convergence and performance with different algorithms.