Understanding and Exploiting Plasticity for Non-stationary Network Resource Adaptation

TL;DR

This work addresses resource adaptation in non-stationary networks by revealing neural plasticity loss as a fundamental bottleneck for learning-based approaches. It introduces Silent Neurons and the Reset Silent Neuron (ReSiN) mechanism, which jointly considers forward activity and backward gradients to preserve plasticity and maintain rapid adaptation. The authors formalize the Silent Neuron framework, prove bidirectional dormancy properties, and demonstrate that resetting silent units improves adaptive video streaming performance, achieving up to 168% higher bitrate and 108% QoE gains while remaining robust in stationary environments. The findings provide a theory-backed pathway for robust, real-time network adaptation in dynamic conditions with practical implications for DRL-based resource management.

Abstract

Adapting to non-stationary network conditions presents significant challenges for resource adaptation. However, current solutions primarily rely on stationary assumptions. While data-driven reinforcement learning approaches offer promising solutions for handling network dynamics, our systematic investigation reveals a critical limitation: neural networks suffer from plasticity loss, significantly impeding their ability to adapt to evolving network conditions. Through theoretical analysis of neural propagation mechanisms, we demonstrate that existing dormant neuron metrics inadequately characterize neural plasticity loss. To address this limitation, we have developed the Silent Neuron theory, which provides a more comprehensive framework for understanding plasticity degradation. Based on these theoretical insights, we propose the Reset Silent Neuron (ReSiN), which preserves neural plasticity through strategic neuron resets guided by both forward and backward propagation states. In our implementation of an adaptive video streaming system, ReSiN has shown significant improvements over existing solutions, achieving up to 168% higher bitrate and 108% better quality of experience (QoE) while maintaining comparable smoothness. Furthermore, ReSiN consistently outperforms in stationary environments, demonstrating its robust adaptability across different network conditions.

Figures (11)

• Figure 1: Performance comparison of bitrate rewards in adaptive video streaming achieved by PPO under different bandwidth conditions.
• Figure 2: Evolution of neural network plasticity loss.
• Figure 3: Performance comparison of PPO under different network conditions in terms of QoE metrics. The figure shows the learning curves of the total QoE reward and its components.
• Figure 4: System metrics under different plasticity maintenance strategies in non-stationary environments are compared in the figure. The strategies include standard PPO (PPO-NS), PPO with output-based reset using dormant neuron detection (PPO-NS-OR), and our proposed method of PPO with intersection-based reset using the proposed Silent Neuron criterion (Ours).
• Figure 5: The activity patterns of neurons across network layers and components, such as the ratio of dormant neurons (Layer-Dormant, LD), the persistence of dormant neurons (DormantOverLeap, LDO), the ratio of zero gradients (ZeroGradient, LZG), and the persistence of zero gradients (ZeroGradientOverLeap, LZGO) are examined for both policy (Policy-0, Policy-1) and value (Value-0, Value-1) networks.

Theorems & Definitions (18)

• Definition 2.1: Overlap Coefficient for Neuron
• Definition 4.1: Dormant Neuron
• Lemma 4.1: Forward Dormancy Lemma
• proof
• Lemma 4.2: Forward-to-Backward Dormancy Lemma
• proof
• Lemma 4.3: Backward Dormancy Lemma
• proof
• Theorem 4.4: Bidirectional Dormancy Characterization
• proof