Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NU-Class Net: A Novel Approach for Video Quality Enhancement

Parham Zilouchian Moghaddam, Mehdi Modarressi, Mohammad Amin Sadeghi

TL;DR

The paper addresses the challenge of maintaining video quality at low bitrates in resource-constrained IoT edge devices by proposing NU-Class Net, a post-decoder neural module that enhances decoded frames without changing the underlying codec. Built on a U-Net–style encoder-decoder, it predicts frame residuals to compensate for compression artifacts, and introduces Sequential and Diffusion variants to improve temporal consistency and progressive refinement. The approach is codec-agnostic and demonstrated to significantly improve MAE-based perceptual quality while enabling substantial bitrate reductions and lower encoder complexity. The results suggest practical impact for edge devices, offering energy and bandwidth savings with broad compatibility, and point to future work in performance optimization and hardware acceleration.

Abstract

Video content has experienced a surge in popularity, asserting its dominance over internet traffic and Internet of Things (IoT) networks. Video compression has long been regarded as the primary means of efficiently managing the substantial multimedia traffic generated by video-capturing devices. Nevertheless, video compression algorithms entail significant computational demands in order to achieve substantial compression ratios. This complexity presents a formidable challenge when implementing efficient video coding standards in resource-constrained embedded systems, such as IoT edge node cameras. To tackle this challenge, this paper introduces NU-Class Net, an innovative deep-learning model designed to mitigate compression artifacts stemming from lossy compression codecs. This enhancement significantly elevates the perceptible quality of low-bit-rate videos. By employing the NU-Class Net, the video encoder within the video-capturing node can reduce output quality, thereby generating low-bit-rate videos and effectively curtailing both computation and bandwidth requirements at the edge. On the decoder side, which is typically less encumbered by resource limitations, NU-Class Net is applied after the video decoder to compensate for artifacts and approximate the quality of the original video. Experimental results affirm the efficacy of the proposed model in enhancing the perceptible quality of videos, especially those streamed at low bit rates.

NU-Class Net: A Novel Approach for Video Quality Enhancement

TL;DR

The paper addresses the challenge of maintaining video quality at low bitrates in resource-constrained IoT edge devices by proposing NU-Class Net, a post-decoder neural module that enhances decoded frames without changing the underlying codec. Built on a U-Net–style encoder-decoder, it predicts frame residuals to compensate for compression artifacts, and introduces Sequential and Diffusion variants to improve temporal consistency and progressive refinement. The approach is codec-agnostic and demonstrated to significantly improve MAE-based perceptual quality while enabling substantial bitrate reductions and lower encoder complexity. The results suggest practical impact for edge devices, offering energy and bandwidth savings with broad compatibility, and point to future work in performance optimization and hardware acceleration.

Abstract

Video content has experienced a surge in popularity, asserting its dominance over internet traffic and Internet of Things (IoT) networks. Video compression has long been regarded as the primary means of efficiently managing the substantial multimedia traffic generated by video-capturing devices. Nevertheless, video compression algorithms entail significant computational demands in order to achieve substantial compression ratios. This complexity presents a formidable challenge when implementing efficient video coding standards in resource-constrained embedded systems, such as IoT edge node cameras. To tackle this challenge, this paper introduces NU-Class Net, an innovative deep-learning model designed to mitigate compression artifacts stemming from lossy compression codecs. This enhancement significantly elevates the perceptible quality of low-bit-rate videos. By employing the NU-Class Net, the video encoder within the video-capturing node can reduce output quality, thereby generating low-bit-rate videos and effectively curtailing both computation and bandwidth requirements at the edge. On the decoder side, which is typically less encumbered by resource limitations, NU-Class Net is applied after the video decoder to compensate for artifacts and approximate the quality of the original video. Experimental results affirm the efficacy of the proposed model in enhancing the perceptible quality of videos, especially those streamed at low bit rates.
Paper Structure (14 sections, 3 equations, 12 figures, 1 table)

This paper contains 14 sections, 3 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Integrating deep learning modules into codecs
  4. Image enhancement using deep learning
  5. Exploring Video Synthesis through Generative Models
  6. Proposed Method
  7. System Design
  8. Architectural Design of the NU-Class Net Model
  9. Sequential NU-Class Net
  10. Diffusion NU-Class Net
  11. Exploring Generalizability in NU-Class Net
  12. Implementation
  13. Experiments & Results
  14. Conclusion

Figures (12)

  • Figure 1: Schematic Representation of the Comprehensive Application Process of the NU-Class Model
  • Figure 2: How Base NU-Class Net performs.
  • Figure 3: Detailed Architecture of the NU-Class Net, Illustrating Layers, Nodes, and Connectivity Patterns.
  • Figure 4: Internal Composition of the NU-Block, Featuring Sub-components within Decoder and Encoder Structures.
  • Figure 5: Schematic Illustration of the Inner Structure and Operational Flow within the NU-Block Residual.
  • ...and 7 more figures