Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Security and Real-time FPGA integration for Learned Image Compression

Alaa Mazouz, Carl De Sousa Tria, Sumanta Chaudhuri, Attilio Fiandrotti, Marco Cagnanzzo, Mihai Mitrea, Enzo Tartaglione

TL;DR

This work tackles the challenge of real-time, secure Learned Image Compression (LIC) on FPGA hardware by proposing an integrated workflow that combines iterative channel pruning, quantization, and a novel Quantization Aware Watermarking (QAW) technique with a Digital Rights Management (DRM) scheme. By embedding the watermark during quantization and encrypting the watermarked weights, the approach achieves robust traceability with negligible impact on rate–distortion performance and energy, enabling secure deployment on two FPGA platforms. Empirical results demonstrate real-time encoding across HD to UHD resolutions with favorable latency and power characteristics, while a detailed layer-wise analysis reveals that watermarking is most challenging in the final layer, with Layer 2 offering the best RD/robustness trade-off. The framework is implemented via Xilinx Vitis-AI and AES-256, and is designed to be adaptable to other accelerators, supporting practical secure LIC deployment at the edge.

Abstract

Learnable Image Compression (LIC) has proven capable of outperforming standardized video codecs in compression efficiency. However, achieving both real-time and secure LIC operations on hardware presents significant conceptual and methodological challenges. The present work addresses these challenges by providing an integrated workflow and platform for training, securing, and deploying LIC models on hardware. To this end, a hardware-friendly LIC model is obtained by iteratively pruning and quantizing the model within a standard end-to-end learning framework. Notably, we introduce a novel Quantization-Aware Watermarking (QAW) technique, where the model is watermarked during quantization using a joint loss function, ensuring robust security without compromising model performance. The watermarked weights are then public-key encrypted, guaranteeing both content protection and user traceability. Experimental results across different FPGA platforms evaluate real-time performance, latency, energy consumption, and compression efficiency. The findings highlight that the watermarking and encryption processes maintain negligible impact on compression efficiency (average of -0.4 PSNR) and energy consumption (average of +2%), while still meeting real-time constraints and preserving security properties.

Security and Real-time FPGA integration for Learned Image Compression

TL;DR

This work tackles the challenge of real-time, secure Learned Image Compression (LIC) on FPGA hardware by proposing an integrated workflow that combines iterative channel pruning, quantization, and a novel Quantization Aware Watermarking (QAW) technique with a Digital Rights Management (DRM) scheme. By embedding the watermark during quantization and encrypting the watermarked weights, the approach achieves robust traceability with negligible impact on rate–distortion performance and energy, enabling secure deployment on two FPGA platforms. Empirical results demonstrate real-time encoding across HD to UHD resolutions with favorable latency and power characteristics, while a detailed layer-wise analysis reveals that watermarking is most challenging in the final layer, with Layer 2 offering the best RD/robustness trade-off. The framework is implemented via Xilinx Vitis-AI and AES-256, and is designed to be adaptable to other accelerators, supporting practical secure LIC deployment at the edge.

Abstract

Learnable Image Compression (LIC) has proven capable of outperforming standardized video codecs in compression efficiency. However, achieving both real-time and secure LIC operations on hardware presents significant conceptual and methodological challenges. The present work addresses these challenges by providing an integrated workflow and platform for training, securing, and deploying LIC models on hardware. To this end, a hardware-friendly LIC model is obtained by iteratively pruning and quantizing the model within a standard end-to-end learning framework. Notably, we introduce a novel Quantization-Aware Watermarking (QAW) technique, where the model is watermarked during quantization using a joint loss function, ensuring robust security without compromising model performance. The watermarked weights are then public-key encrypted, guaranteeing both content protection and user traceability. Experimental results across different FPGA platforms evaluate real-time performance, latency, energy consumption, and compression efficiency. The findings highlight that the watermarking and encryption processes maintain negligible impact on compression efficiency (average of -0.4 PSNR) and energy consumption (average of +2%), while still meeting real-time constraints and preserving security properties.

Paper Structure

This paper contains 19 sections, 6 equations, 10 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Learnable Image Compression
  4. Neural Network Watermarking
  5. Proposed Method
  6. Channel Pruning
  7. Quantization Aware Watermarking
  8. Weight quantization
  9. Inserting the Watermarking
  10. Hardware Model Compilation and Deployment
  11. Secure Deployment and Digital Rights Management
  12. Experimental Evaluation
  13. Training the Model
  14. Pruning the Model
  15. Jointly Watermarking and Quantization the Model
  16. ...and 4 more sections

Figures (10)

  • Figure 1: The reference Learned Image Compression (LIC) model balle_end--end_2017 we consider in this work, N is set to 192 and the arrows describe the downsampling and upsampling ratio
  • Figure 2: The proposed workflow for training, optimizing, watermarking, and deploying LIC models on hardware is comprehensive yet abstracts hardware-specific compilation through Xilinx VITIS-AI APIs, ensuring accessibility for non-hardware experts.
  • Figure 3: Quantization Aware Training process dataflow
  • Figure 4: Hardware architecture of the overall Deep learning Processing Unit
  • Figure 5: The proposed DRM scheme
  • ...and 5 more figures