Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReinWiFi: Application-Layer QoS Optimization of WiFi Networks with Reinforcement Learning

Qianren Li, Bojie Lv, Yuncong Hong, Rui Wang

TL;DR

This work addresses the challenge of optimizing application-layer QoS in WiFi networks under unknown interference and vendor-dependent MAC behavior. It proposes ReinWiFi, a reinforcement-learning scheduling framework that jointly tunes access-window settings and application-layer throughput limits to improve both delay-sensitive RTTs and file-delivery throughput, even when PHY/MAC states are not observable. A novel Q-network, augmented by offline imitation learning and performance-region quantization, estimates the value of scheduling decisions; offline training with imitators is followed by online refinement, enabling adaptation to changing traffic and interference. Experiments on a real WiFi testbed demonstrate substantial gains over IEEE 802.11e EDCA, with further improvements from online training, highlighting the practical viability of RL-based application-layer QoS optimization in heterogeneous and dynamic wireless environments.

Abstract

The enhanced distributed channel access (EDCA) mechanism is used in current wireless fidelity (WiFi) networks to support priority requirements of heterogeneous applications. However, the EDCA mechanism can not adapt to particular quality-of-service (QoS) objective, network topology, and interference level. In this paper, a novel reinforcement-learning-based scheduling framework is proposed and implemented to optimize the application-layer quality-of-service (QoS) of a WiFi network with commercial adapters and unknown interference. Particularly, application-layer tasks of file delivery and delay-sensitive communication are jointly scheduled by adjusting the contention window sizes and application-layer throughput limitation, such that the throughput of the former and the round trip time of the latter can be optimized. Due to the unknown interference and vendor-dependent implementation of the WiFi adapters, the relation between the scheduling policy and the system QoS is unknown. Hence, a reinforcement learning method is proposed, in which a novel Q-network is trained to map from the historical scheduling parameters and QoS observations to the current scheduling action. It is demonstrated on a testbed that the proposed framework can achieve a significantly better performance than the EDCA mechanism.

ReinWiFi: Application-Layer QoS Optimization of WiFi Networks with Reinforcement Learning

TL;DR

This work addresses the challenge of optimizing application-layer QoS in WiFi networks under unknown interference and vendor-dependent MAC behavior. It proposes ReinWiFi, a reinforcement-learning scheduling framework that jointly tunes access-window settings and application-layer throughput limits to improve both delay-sensitive RTTs and file-delivery throughput, even when PHY/MAC states are not observable. A novel Q-network, augmented by offline imitation learning and performance-region quantization, estimates the value of scheduling decisions; offline training with imitators is followed by online refinement, enabling adaptation to changing traffic and interference. Experiments on a real WiFi testbed demonstrate substantial gains over IEEE 802.11e EDCA, with further improvements from online training, highlighting the practical viability of RL-based application-layer QoS optimization in heterogeneous and dynamic wireless environments.

Abstract

The enhanced distributed channel access (EDCA) mechanism is used in current wireless fidelity (WiFi) networks to support priority requirements of heterogeneous applications. However, the EDCA mechanism can not adapt to particular quality-of-service (QoS) objective, network topology, and interference level. In this paper, a novel reinforcement-learning-based scheduling framework is proposed and implemented to optimize the application-layer quality-of-service (QoS) of a WiFi network with commercial adapters and unknown interference. Particularly, application-layer tasks of file delivery and delay-sensitive communication are jointly scheduled by adjusting the contention window sizes and application-layer throughput limitation, such that the throughput of the former and the round trip time of the latter can be optimized. Due to the unknown interference and vendor-dependent implementation of the WiFi adapters, the relation between the scheduling policy and the system QoS is unknown. Hence, a reinforcement learning method is proposed, in which a novel Q-network is trained to map from the historical scheduling parameters and QoS observations to the current scheduling action. It is demonstrated on a testbed that the proposed framework can achieve a significantly better performance than the EDCA mechanism.
Paper Structure (14 sections, 21 equations, 3 figures, 1 table)

This paper contains 14 sections, 21 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model
  3. Deployment Scenario
  4. Task Queuing Model
  5. Scheduling Model
  6. Problem Formulation
  7. Q-Network for Online Scheduling
  8. Performance Region Quantization
  9. Q-Network Structure
  10. Hybrid Q-Learning
  11. Offline Imitation Learning and Q-Network Training
  12. Online Q-network Training
  13. Experiments
  14. Conclusion

Figures (3)

  • Figure 1: Illustration of testbed.
  • Figure 2: Performance comparison in scenarios $1 \sim 5$.
  • Figure 3: Performance comparison in scenarios $6 \sim 11$.

Theorems & Definitions (4)

  • Definition 1: System State
  • Definition 2: Scheduling Action and Policy
  • Remark 1
  • Definition 3: Extended System State