Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantized neural network for complex hologram generation

Yutaka Endo, Minoru Oikawa, Timothy D. Wilkinson, Tomoyoshi Shimobaba, Tomoyoshi Ito

TL;DR

This study built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8) and shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size and increasing the speed fourfold.

Abstract

Computer-generated holography (CGH) is a promising technology for augmented reality displays, such as head-mounted or head-up displays. However, its high computational demand makes it impractical for implementation. Recent efforts to integrate neural networks into CGH have successfully accelerated computing speed, demonstrating the potential to overcome the trade-off between computational cost and image quality. Nevertheless, deploying neural network-based CGH algorithms on computationally limited embedded systems requires more efficient models with lower computational cost, memory footprint, and power consumption. In this study, we developed a lightweight model for complex hologram generation by introducing neural network quantization. Specifically, we built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8). Our performance evaluation shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size by approximately 70% and increasing the speed fourfold. Additionally, we implemented the INT8 model on a system-on-module to demonstrate its deployability on embedded platforms and high power efficiency.

Quantized neural network for complex hologram generation

TL;DR

This study built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8) and shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size and increasing the speed fourfold.

Abstract

Computer-generated holography (CGH) is a promising technology for augmented reality displays, such as head-mounted or head-up displays. However, its high computational demand makes it impractical for implementation. Recent efforts to integrate neural networks into CGH have successfully accelerated computing speed, demonstrating the potential to overcome the trade-off between computational cost and image quality. Nevertheless, deploying neural network-based CGH algorithms on computationally limited embedded systems requires more efficient models with lower computational cost, memory footprint, and power consumption. In this study, we developed a lightweight model for complex hologram generation by introducing neural network quantization. Specifically, we built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8). Our performance evaluation shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size by approximately 70% and increasing the speed fourfold. Additionally, we implemented the INT8 model on a system-on-module to demonstrate its deployability on embedded platforms and high power efficiency.
Paper Structure (15 sections, 5 equations, 4 figures, 2 tables)

This paper contains 15 sections, 5 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Complex hologram generation and tensor holography
  4. Model architecture and training
  5. Neural network quantization
  6. Results
  7. Hologram quality
  8. Reconstructed images
  9. Model size, latency, and frame rate
  10. Deployment on embedded platforms
  11. Discussion
  12. Training and quantization settings
  13. Image quality evaluation
  14. Model implementation
  15. Conclusion

Figures (4)

  • Figure 1: Model architecture for computing complex holograms from RGB-D images. The residual blocks consist of two convolution layers with batch normalization (BN) and ReLU6 activation functions. The output of the final residual block and the skip-connected input are concatenated in the depthwise separable convolution block.
  • Figure 2: Histograms of two activations that are concatenated in both the original and refined models.
  • Figure 3: Hologram quality of the original tensor holography model and our refined model in FP32 and INT8 with PTDQ, PTSQ, and QAT. PSNR and SSIM are averaged over the test dataset.
  • Figure 4: Reconstructed images from complex holograms computed by original tensor holography model and our refined model in FP32 and INT8 with PTDQ, PTSQ, and QAT. The PSNR and SSIM are computed between the reconstructed images from the INT8 and FP32 models. © 2008, Blender Foundation / www.bigbuckbunny.org.