Integrated Sensing, Computing, Communication, and Control for Time-Sequence-Based Semantic Communications

TL;DR

This work tackles ultra-reliable low-latency wireless control in IIoT by introducing time-sequence-based semantic communications within an Integrated Sensing, Computing, Communication, and Control (ISC3) framework. It leverages a semantic feature extractor (SFE) at the transmitter and a semantic feature reconstructor (SFR) at the receiver, guided by mutual information estimates from a Mutual Information Neural Estimator (MINE), to dynamically update transmissions and predict control commands when not transmitted. A Hybrid Reward Multi-Agent Deep Reinforcement Learning (HR-MADRL) framework trains discrete transmission and continuous gain policies, balancing communication overhead with control accuracy via a Hubber loss-based reward and long-term Q-learning objectives. Experiments on a teleoperation setup with Geomagic Touch X and a Panda robot show substantial reductions in duty cycle while maintaining or improving control performance, demonstrating practical gains for URLLC-enabled WCSs. The approach suggests strong potential for scalable, task-oriented wireless control in industrial settings and beyond, with future work extending to physical-layer integration and higher-frequency sensing to further improve performance.$

Abstract

In the upcoming industrial internet of things (IIoT) era, a surge of task-oriented applications will rely on real-time wireless control systems (WCSs). For these systems, ultra-reliable and low-latency wireless communication will be crucial to ensure the timely transmission of control information. To achieve this purpose, we propose a novel time-sequence-based semantic communication paradigm, where an integrated sensing, computing, communication, and control (ISC3) architecture is developed to make sensible semantic inference (SI) for the control information over time sequences, enabling adaptive control of the robot. However, due to the causal correlations in the time sequence, the control information does not present the Markov property. To address this challenge, we compute the mutual information of the control information sensed at the transmitter (Tx) over different time and identify their temporal semantic correlation via a semantic feature extractor (SFE) module. By this means, highly correlated information transmission can be avoided, thus greatly reducing the communication overhead. Meanwhile, a semantic feature reconstructor (SFR) module is employed at the receiver (Rx) to reconstruct the control information based on the previously received one if the information transmission is not activated at the Tx. Furthermore, a control gain policy is also employed at the Rx to adaptively adjust the control gain for the controlled target based on several practical aspects such as the quality of the information transmission from the Tx to the Rx. We design the neural network structures of the above modules/policies and train their parameters by a novel hybrid reward multi-agent deep reinforcement learning framework. On-site experiments are conducted to evaluate the performance of our proposed method in practice, which shows significant gains over other baseline schemes.

Figures (14)

• Figure 1: Comparison between the traditional WCS and the proposed time-sequence-based SC-enabled WCS.
• Figure 2: Proposed WCS with ISC3.
• Figure 3: Proposed LSTM encoder-decoder structure.
• Figure 4: Proposed HR-MADRL architecture.
• Figure 5: Proposed HR-MADRL learning structure.

Theorems & Definitions (1)

• proof