Auction-Based RIS Allocation With DRL: Controlling the Cost-Performance Trade-Off

TL;DR

This work demonstrates that reinforcement learning (RL)-based bidding significantly outperforms heuristic strategies, achieving optimal trade-offs between cost and spectral efficiency and introduces a tunable parameter that governs the bidding aggressiveness of RL agents, enabling a flexible control of the trade-off between network performance and expenditure.

Abstract

We study the allocation of reconfigurable intelligent surfaces (RISs) in a multi-cell wireless network, where base stations compete for control of shared RIS units deployed at the cell edges. These RISs, provided by an independent operator, are dynamically leased to the highest bidder using a simultaneously ascending auction format. Each base station estimates the utility of acquiring additional RISs based on macroscopic channel parameters, enabling a scalable and low-overhead allocation mechanism. To optimize the bidding behavior, we integrate deep reinforcement learning (DRL) agents that learn to maximize performance while adhering to budget constraints. Through simulations in clustered cell-edge environments, we demonstrate that reinforcement learning (RL)-based bidding significantly outperforms heuristic strategies, achieving optimal trade-offs between cost and spectral efficiency. Furthermore, we introduce a tunable parameter that governs the bidding aggressiveness of RL agents, enabling a flexible control of the trade-off between network performance and expenditure. Our results highlight the potential of combining auction-based allocation with adaptive RL mechanisms for efficient and fair utilization of RISs in next-generation wireless networks.

Figures (6)

• Figure 1: System workflow of the proposed auction-based RIS allocation framework. Macroscopic SINR and utility estimates are used as inputs to a DRL-based bidding policy, which generates bids for the auction mechanism. The auction iterates over bidding rounds until a termination condition is met, after which data transmission is performed.
• Figure 2: Simulation geometry with two base stations (BS) at opposite ends, clustered users (UE), and RISs positioned along the cell boundary. The figure shows an example allocation of RISs obtained using a reinforcement learning agent with bid intensity parameter $\beta=3$.
• Figure 3: Accuracy of the macroscopic SINR estimation as a function of the number of BS antennas $M_{\mathrm{BS}}$. The figure reports the mean, median, and 90th percentile of the absolute error between the estimated and true SINR values.
• Figure 4: Training convergence of the PPO-based bidding agent with $\beta=2$. The figure shows the evolution of the training reward over environment interaction steps, with a moving-average smoothed curve (window size 5).
• Figure 5: Performance comparison of heuristics and RL-based models using different bid intensity ($\beta$) values. The figure illustrates the trade-off between cost and achievable rate.