Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive RAN Slicing Control via Reward-Free Self-Finetuning Agents

Yuanhao Li, Haozhe Wang, Geyong Min, Nektarios Georgalas, Wang Miao

TL;DR

This paper proposes a novel self-finetuning framework that enables agentic systems to learn continuously through direct interaction with the environment, bypassing the need for handcrafted rewards.

Abstract

The integration of Generative AI models into AI-native network systems offers a transformative path toward achieving autonomous and adaptive control. However, the application of such models to continuous control tasks is impeded by intrinsic architectural limitations, including finite context windows, the lack of explicit reward signals, and the degradation of the long context. This paper posits that the key to unlocking robust continuous control is enabling agents to internalize experience by distilling it into their parameters, rather than relying on prompt-based memory. To this end, we propose a novel self-finetuning framework that enables agentic systems to learn continuously through direct interaction with the environment, bypassing the need for handcrafted rewards. Our framework implements a bi-perspective reflection mechanism that generates autonomous linguistic feedback to construct preference datasets from interaction history. A subsequent preference-based fine-tuning process distills long-horizon experiences into the model's parameters. We evaluate our approach on a dynamic Radio Access Network (RAN) slicing task, a challenging multi-objective control problem that requires the resolution of acute trade-offs between spectrum efficiency, service quality, and reconfiguration stability under volatile network conditions. Experimental results show that our framework outperforms standard Reinforcement Learning (RL) baselines and existing Large Language Model (LLM)-based agents in sample efficiency, stability, and multi-metric optimization. These findings demonstrate the potential of self-improving generative agents for continuous control tasks, paving the way for future AI-native network infrastructure.

Adaptive RAN Slicing Control via Reward-Free Self-Finetuning Agents

TL;DR

This paper proposes a novel self-finetuning framework that enables agentic systems to learn continuously through direct interaction with the environment, bypassing the need for handcrafted rewards.

Abstract

The integration of Generative AI models into AI-native network systems offers a transformative path toward achieving autonomous and adaptive control. However, the application of such models to continuous control tasks is impeded by intrinsic architectural limitations, including finite context windows, the lack of explicit reward signals, and the degradation of the long context. This paper posits that the key to unlocking robust continuous control is enabling agents to internalize experience by distilling it into their parameters, rather than relying on prompt-based memory. To this end, we propose a novel self-finetuning framework that enables agentic systems to learn continuously through direct interaction with the environment, bypassing the need for handcrafted rewards. Our framework implements a bi-perspective reflection mechanism that generates autonomous linguistic feedback to construct preference datasets from interaction history. A subsequent preference-based fine-tuning process distills long-horizon experiences into the model's parameters. We evaluate our approach on a dynamic Radio Access Network (RAN) slicing task, a challenging multi-objective control problem that requires the resolution of acute trade-offs between spectrum efficiency, service quality, and reconfiguration stability under volatile network conditions. Experimental results show that our framework outperforms standard Reinforcement Learning (RL) baselines and existing Large Language Model (LLM)-based agents in sample efficiency, stability, and multi-metric optimization. These findings demonstrate the potential of self-improving generative agents for continuous control tasks, paving the way for future AI-native network infrastructure.
Paper Structure (19 sections, 14 equations, 3 figures, 3 tables, 2 algorithms)

This paper contains 19 sections, 14 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. RL for AI-Native Networking
  4. LLM Agent
  5. RAN Slicing Resource Management
  6. System Model
  7. Problem Definition
  8. Methodology
  9. Reflective Markov Decision Process
  10. Actor-Reflector Framework
  11. Actor
  12. Reflector
  13. Bi-Perspective Reflection
  14. Refine-from-Reflection (RFR) Fine-tuning framework
  15. Experiment
  16. ...and 4 more sections

Figures (3)

  • Figure 1: This figure compares three control algorithms (RL Actor-Critic, Reflexion, and Self-Finetuning), each organized into four key functional modules, color-coded for clarity: action-generating Actor (gray), interactive Environment (green), performance-evaluating module (blue), and Actor updating mechanism (yellow). RL updates the Actor via Advantage-based loss; Reflexion leverages self-reflection and a trajectory database to inject insights and history into the Actor’s prompt; Self-Finetuning generates training data through Reflection and directly improves the Actor via KTO Preference Finetuning.
  • Figure 2: Performance comparison of RL baselines, Reflexion, and our Self-Finetuning method across multiple objectives
  • Figure 3: Training dynamics of Self-Finetuning using one trajectory. (a) illustrates improved sla stability, 33% fewer reconfigurations, and higher spectral efficiency; (b–g) show KTO reward evolution in each iteration; (h) Reward convergence over KTO iterations, as chosen and rejected rewards both approach zero, indicating policy stabilization.