Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual-Mind World Models: A General Framework for Learning in Dynamic Wireless Networks

Lingyi Wang, Rashed Shelim, Walid Saad, Naren Ramakrishnan

TL;DR

This work tackles data inefficiency and short-sighted planning in reinforcement learning for dynamic wireless networks by introducing Dual-Mind World Models (DMWM), which fuse a pattern-driven System 1 based on a recurrent state-space model (RSSM) with a logic-driven System 2 using neural-symbolic reasoning (LINN). A logic-enhanced ELBO (LE-ELBO) ties imaginations to logical consistency, enabling reliable long-horizon planning of link scheduling in a mmWave V2X CAoI minimization problem. The authors validate their approach on a realistic Sionna-based simulator augmented with Blender and ArcGIS, reporting significant gains in data efficiency and CAoI reduction (up to around 32% over strong baselines) and strong generalization to unseen road scenes and network configurations. The results advocate for a practical, differentiable imagination-driven planning paradigm that integrates physics-based network dynamics with symbolic logic, offering a path toward scalable, robust wireless control under complex dynamics.

Abstract

Despite the popularity of reinforcement learning (RL) in wireless networks, existing approaches that rely on model-free RL (MFRL) and model-based RL (MBRL) are data inefficient and short-sighted. Such RL-based solutions cannot generalize to novel network states since they capture only statistical patterns rather than the underlying physics and logic from wireless data. These limitations become particularly challenging in complex wireless networks with high dynamics and long-term planning requirements. To address these limitations, in this paper, a novel dual-mind world model-based learning framework is proposed with the goal of optimizing completeness-weighted age of information (CAoI) in a challenging mmWave V2X scenario. Inspired by cognitive psychology, the proposed dual-mind world model encompasses a pattern-driven System 1 component and a logic-driven System 2 component to learn dynamics and logic of the wireless network, and to provide long-term link scheduling over reliable imagined trajectories. Link scheduling is learned through end-to-end differentiable imagined trajectories with logical consistency over an extended horizon rather than relying on wireless data obtained from environment interactions. Moreover, through imagination rollouts, the proposed world model can jointly reason network states and plan link scheduling. During intervals without observations, the proposed method remains capable of making efficient decisions. Extensive experiments are conducted on a realistic simulator based on Sionna with real-world physical channel, ray-tracing, and scene objects with material properties. Simulation results show that the proposed world model achieves a significant improvement in data efficiency and achieves strong generalization and adaptation to unseen environments, compared to the state-of-the-art RL baselines, and the world model approach with only System 1.

Dual-Mind World Models: A General Framework for Learning in Dynamic Wireless Networks

TL;DR

This work tackles data inefficiency and short-sighted planning in reinforcement learning for dynamic wireless networks by introducing Dual-Mind World Models (DMWM), which fuse a pattern-driven System 1 based on a recurrent state-space model (RSSM) with a logic-driven System 2 using neural-symbolic reasoning (LINN). A logic-enhanced ELBO (LE-ELBO) ties imaginations to logical consistency, enabling reliable long-horizon planning of link scheduling in a mmWave V2X CAoI minimization problem. The authors validate their approach on a realistic Sionna-based simulator augmented with Blender and ArcGIS, reporting significant gains in data efficiency and CAoI reduction (up to around 32% over strong baselines) and strong generalization to unseen road scenes and network configurations. The results advocate for a practical, differentiable imagination-driven planning paradigm that integrates physics-based network dynamics with symbolic logic, offering a path toward scalable, robust wireless control under complex dynamics.

Abstract

Despite the popularity of reinforcement learning (RL) in wireless networks, existing approaches that rely on model-free RL (MFRL) and model-based RL (MBRL) are data inefficient and short-sighted. Such RL-based solutions cannot generalize to novel network states since they capture only statistical patterns rather than the underlying physics and logic from wireless data. These limitations become particularly challenging in complex wireless networks with high dynamics and long-term planning requirements. To address these limitations, in this paper, a novel dual-mind world model-based learning framework is proposed with the goal of optimizing completeness-weighted age of information (CAoI) in a challenging mmWave V2X scenario. Inspired by cognitive psychology, the proposed dual-mind world model encompasses a pattern-driven System 1 component and a logic-driven System 2 component to learn dynamics and logic of the wireless network, and to provide long-term link scheduling over reliable imagined trajectories. Link scheduling is learned through end-to-end differentiable imagined trajectories with logical consistency over an extended horizon rather than relying on wireless data obtained from environment interactions. Moreover, through imagination rollouts, the proposed world model can jointly reason network states and plan link scheduling. During intervals without observations, the proposed method remains capable of making efficient decisions. Extensive experiments are conducted on a realistic simulator based on Sionna with real-world physical channel, ray-tracing, and scene objects with material properties. Simulation results show that the proposed world model achieves a significant improvement in data efficiency and achieves strong generalization and adaptation to unseen environments, compared to the state-of-the-art RL baselines, and the world model approach with only System 1.

Paper Structure

This paper contains 22 sections, 1 theorem, 23 equations, 10 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Works
  4. System Model
  5. Transmission Model
  6. CAoI Metric
  7. CAoI Minimization Problem
  8. Dual-Mind World Model For Long-Term Prediction and Link Scheduling
  9. Pattern-Driven System 1 for Fast Inference
  10. RSSM-Based Pattern Learning
  11. Logic-Driven System 2 for Deep Inference
  12. Neural Network-Based Logic Operations
  13. Proposed Deep Logical Reasoning
  14. Inter-System Signal Mechanism
  15. Learning Link Scheduling in Imagined Trajectories
  16. ...and 7 more sections

Key Result

Theorem 1

Considering the logical signals from the System 2 component to the System 1 component, the LE-ELBO of the imagination loss can be bounded by (eq:bound), where $q_1=q_\varphi\left(z_t \mid o_{\leq t}, a_{<t}\right)$, $q_2=q_\varphi\left(z_{t-1} \mid o_{\leq t-1}, a_{< t-1}\right)$, and the prior stat

Figures (10)

  • Figure 1: Illustration of the use of a world model for learning and optimization in a mmWave V2X communication network.
  • Figure 2: Learning a world model for V2X communication networks based on the location data and the ray tracing data.
  • Figure 3: The logic-enhanced imagination ability of the proposed dual-mind world model for both policy learning, and joint prediction and scheduling without wireless data.
  • Figure 4: Procedures of the proposed realistic simulator based on Sionna, Blender, ArcGIS, Mitsuba, and plug-in of Blender-OSM and Mitsuba-Blender.
  • Figure 5: The average test rewards of different schemes under limited environment steps.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Theorem 1
  • proof