Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distillation-Enabled Knowledge Alignment for Generative Semantic Communications of AIGC Images

Jingzhi Hu, Geoffrey Ye Li

TL;DR

This work tackles knowledge alignment (KA) in generative semantic communications for AIGC images across cloud edge and edge user. It introduces DeKA-g, a distillation-enabled KA framework that splits KA into generation (G-KA) and transmission (T-KA) tasks, solved via metaword-aided KD (MAKD) and condition-aware low-rank adaptation (CALA). MAKD uses a trainable metaword to align prompt interpretation between cloud and edge GAI, distilling into compact LoRA updates; CALA employs a soft gating mechanism to adapt LoRA updates to diverse transmission conditions, avoiding interference. Empirical results show MA KD yields a 44% increase in DINO+CLIP consistency and CALA achieves up to 6.5 dB PSNR gains over baselines while updating only a tiny fraction of parameters, enabling scalable, bandwidth-efficient AIGC provisioning in edge networks.

Abstract

Due to the surging amount of AI-generated images, its provisioning to edges and mobile users from the cloud incurs substantial traffic on networks. Generative semantic communication (GSC) offers a promising solution by transmitting highly compact information, i.e., prompt text and latent representations, instead of high-dimensional image data. However, GSC relies on the alignment between the knowledge in the cloud generative AI (GAI) and that possessed by the edges and users, and between the knowledge for wireless transmission and that of actual channels, which remains challenging. In this paper, we propose DeKA-g, a distillation-enabled knowledge alignment algorithm for GSC systems. The core idea is to distill the image generation knowledge from the cloud-GAI into low-rank matrices, which can be incorporated by the edge and used to adapt the transmission knowledge to diverse wireless channel conditions. DeKA-g comprises two novel methods: metaword-aided knowledge distillation (MAKD) and condition-aware low-rank adaptation (CALA). For MAKD, an optimized metaword is employed to enhance the efficiency of knowledge distillation, while CALA enables efficient adaptation to diverse rate requirements and channel conditions. From simulation results, DeKA-g improves the consistency between the edge-generated images and the cloud-generated ones by 44% and enahnces the average transmission quality in terms of PSNR by 6.5 dB over the baselines without knowledge alignment.

Distillation-Enabled Knowledge Alignment for Generative Semantic Communications of AIGC Images

TL;DR

This work tackles knowledge alignment (KA) in generative semantic communications for AIGC images across cloud edge and edge user. It introduces DeKA-g, a distillation-enabled KA framework that splits KA into generation (G-KA) and transmission (T-KA) tasks, solved via metaword-aided KD (MAKD) and condition-aware low-rank adaptation (CALA). MAKD uses a trainable metaword to align prompt interpretation between cloud and edge GAI, distilling into compact LoRA updates; CALA employs a soft gating mechanism to adapt LoRA updates to diverse transmission conditions, avoiding interference. Empirical results show MA KD yields a 44% increase in DINO+CLIP consistency and CALA achieves up to 6.5 dB PSNR gains over baselines while updating only a tiny fraction of parameters, enabling scalable, bandwidth-efficient AIGC provisioning in edge networks.

Abstract

Due to the surging amount of AI-generated images, its provisioning to edges and mobile users from the cloud incurs substantial traffic on networks. Generative semantic communication (GSC) offers a promising solution by transmitting highly compact information, i.e., prompt text and latent representations, instead of high-dimensional image data. However, GSC relies on the alignment between the knowledge in the cloud generative AI (GAI) and that possessed by the edges and users, and between the knowledge for wireless transmission and that of actual channels, which remains challenging. In this paper, we propose DeKA-g, a distillation-enabled knowledge alignment algorithm for GSC systems. The core idea is to distill the image generation knowledge from the cloud-GAI into low-rank matrices, which can be incorporated by the edge and used to adapt the transmission knowledge to diverse wireless channel conditions. DeKA-g comprises two novel methods: metaword-aided knowledge distillation (MAKD) and condition-aware low-rank adaptation (CALA). For MAKD, an optimized metaword is employed to enhance the efficiency of knowledge distillation, while CALA enables efficient adaptation to diverse rate requirements and channel conditions. From simulation results, DeKA-g improves the consistency between the edge-generated images and the cloud-generated ones by 44% and enahnces the average transmission quality in terms of PSNR by 6.5 dB over the baselines without knowledge alignment.

Paper Structure

This paper contains 25 sections, 10 equations, 14 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. GSC Systems
  3. Generation Module
  4. Transmission Module
  5. KA Problem Formulation for GSC Systems
  6. Motivation
  7. Problem Formulation
  8. Distillation-Enabled KA Algorithm for GSC systems
  9. Decomposition of KA Problem
  10. MAKD for G-KA
  11. CALA for T-KA
  12. Neural Model Implementation
  13. Generation Module
  14. Transmission Module
  15. Performance Evaluation
  16. ...and 10 more sections

Figures (14)

  • Figure 1: The GSC system for AIGC image provisioning, under aligned and misaligned knowledge.
  • Figure 2: Systematic diagram of the GSC system.
  • Figure 3: Designed neural model of the GSC system.
  • Figure 4: Overall comparison between DeKA-g and benchmark algorithms.
  • Figure 5: Comparison of different G-KA methods in terms of (a) DINO-score and (b) CLIP-score.
  • ...and 9 more figures