End-to-End Generative Semantic Communication Powered by Shared Semantic Knowledge Base

TL;DR

The paper introduces an SKB-enabled end-to-end generative semantic communication framework for images, tackling low testing CR by transmitting only a compact index to convey task-relevant semantics. It uses a shared class-level SKB of attribute vectors ${\bf k}_m \in \mathbb{R}^d$ and a cosine-matching encoder to map images to semantic features $\mathbf{s}$, transmitting the index $\mathbf{v}$ (length $t=1$) to perform image classification at the transmitter and enable controllable generation at the receiver via a hierarchical CVAE with latent $\mathbf{z}$. For generation, when the CR is low ($B<\theta$), only the index is transmitted to produce class-consistent images; when CR is high enough ($B\ge\theta$), latent information $\mathbf{z}$ is also sent to reconstruct the original image, all under AWGN channel conditions. A semantic accuracy metric is proposed to quantify the usefulness of received semantics, complementing traditional image-quality measures. Experiments on the CUB dataset show robust performance and reduced transmission overhead, with higher SKB dimension $d$ improving both classification and generation metrics, and notable resilience in low-SNR regimes compared to JPEG+LDPC baselines and Vanilla SemCom.

Abstract

Semantic communication has drawn substantial attention as a promising paradigm to achieve effective and intelligent communications. However, efficient image semantic communication encounters challenges with a lower testing compression ratio (CR) compared to the training phase. To tackle this issue, we propose an innovative semantic knowledge base (SKB)-enabled generative semantic communication system for image classification and image generation tasks. Specifically, a lightweight SKB, comprising class-level information, is exploited to guide the semantic communication process, which enables us to transmit only the relevant indices. This approach promotes the completion of the image classification task at the source end and significantly reduces the transmission load. Meanwhile, the category-level knowledge in the SKB facilitates the image generation task by allowing controllable generation, making it possible to generate favorable images in resource-constrained scenarios. Additionally, semantic accuracy is introduced as a new metric to validate the performance of semantic transmission powered by the SKB. Evaluation results indicate that the proposed method outperforms the benchmarks and achieves superior performance with minimal transmission overhead, especially in the low SNR regime.

Figures (4)

• Figure 1: The framework of proposed SKB-enabled generative semantic communication system. All ellipses indicate the semantic knowledge base.
• Figure 2: Classification accuracy and semantic accuracy versus SNR.
• Figure 3: Performance comparison of (a) PSNR, (b) SSIM, (c) LPIPS and (d) FID within different SNR. Better performance is indicated by higher values of PSNR and SSIM, and lower values of LPIPS and FID.
• Figure 4: Examples of visual comparison when $\theta$ = 0.021. The first and second rows represent the cases with SNR as 2dB and 10dB, respectively.