Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A New Comprehensive Framework for Multi-Exposure Stereo Coding Utilizing Low Rank Tucker-ALS and 3D-HEVC Techniques

Mansi Sharma, Jyotsana Grover

TL;DR

This work tackles the challenge of efficiently compressing HDR stereo content without direct HDR capture by fusing multiple exposure LDR stereo images. It introduces a Tucker-ALS-based low-rank tensor representation of the exposure-stacked stereo data and encodes the compact representation using 3D-HEVC, allowing bitrate control through core-tensor ranks. The approach jointly investigates color spaces ($Y^{'}C_bC_r$ and IPT) and demonstrates bitrate savings with maintained HDR quality (PSNR, HDR-VDP-2) across multiple scenes. By exploiting intra-frame, inter-view, and inter-component redundancies, the method supports stereo personalization for 3D displays and offers practical benefits for HDR stereo delivery.

Abstract

Display technology must offer high dynamic range (HDR) contrast-based depth induction and 3D personalization simultaneously. Efficient algorithms to compress HDR stereo data is critical. Direct capturing of HDR content is complicated due to the high expense and scarcity of HDR cameras. The HDR 3D images could be generated in low-cost by fusing low-dynamic-range (LDR) images acquired using a stereo camera with various exposure settings. In this paper, an efficient scheme for coding multi-exposure stereo images is proposed based on a tensor low-rank approximation scheme. The multi-exposure fusion can be realized to generate HDR stereo output at the decoder for increased realism and exaggerated binocular 3D depth cues. For exploiting spatial redundancy in LDR stereo images, the stack of multi-exposure stereo images is decomposed into a set of projection matrices and a core tensor following an alternating least squares Tucker decomposition model. The compact, low-rank representation of the scene, thus, generated is further processed by 3D extension of High Efficiency Video Coding standard. The encoding with 3D-HEVC enhance the proposed scheme efficiency by exploiting intra-frame, inter-view and the inter-component redundancies in low-rank approximated representation. We consider constant luminance property of IPT and Y'CbCr color space to precisely approximate intensity prediction and perceptually minimize the encoding distortion. Besides, the proposed scheme gives flexibility to adjust the bitrate of tensor latent components by changing the rank of core tensor and its quantization. Extensive experiments on natural scenes demonstrate that the proposed scheme outperforms state-of-the-art JPEG-XT and 3D-HEVC range coding standards.

A New Comprehensive Framework for Multi-Exposure Stereo Coding Utilizing Low Rank Tucker-ALS and 3D-HEVC Techniques

TL;DR

This work tackles the challenge of efficiently compressing HDR stereo content without direct HDR capture by fusing multiple exposure LDR stereo images. It introduces a Tucker-ALS-based low-rank tensor representation of the exposure-stacked stereo data and encodes the compact representation using 3D-HEVC, allowing bitrate control through core-tensor ranks. The approach jointly investigates color spaces ( and IPT) and demonstrates bitrate savings with maintained HDR quality (PSNR, HDR-VDP-2) across multiple scenes. By exploiting intra-frame, inter-view, and inter-component redundancies, the method supports stereo personalization for 3D displays and offers practical benefits for HDR stereo delivery.

Abstract

Display technology must offer high dynamic range (HDR) contrast-based depth induction and 3D personalization simultaneously. Efficient algorithms to compress HDR stereo data is critical. Direct capturing of HDR content is complicated due to the high expense and scarcity of HDR cameras. The HDR 3D images could be generated in low-cost by fusing low-dynamic-range (LDR) images acquired using a stereo camera with various exposure settings. In this paper, an efficient scheme for coding multi-exposure stereo images is proposed based on a tensor low-rank approximation scheme. The multi-exposure fusion can be realized to generate HDR stereo output at the decoder for increased realism and exaggerated binocular 3D depth cues. For exploiting spatial redundancy in LDR stereo images, the stack of multi-exposure stereo images is decomposed into a set of projection matrices and a core tensor following an alternating least squares Tucker decomposition model. The compact, low-rank representation of the scene, thus, generated is further processed by 3D extension of High Efficiency Video Coding standard. The encoding with 3D-HEVC enhance the proposed scheme efficiency by exploiting intra-frame, inter-view and the inter-component redundancies in low-rank approximated representation. We consider constant luminance property of IPT and Y'CbCr color space to precisely approximate intensity prediction and perceptually minimize the encoding distortion. Besides, the proposed scheme gives flexibility to adjust the bitrate of tensor latent components by changing the rank of core tensor and its quantization. Extensive experiments on natural scenes demonstrate that the proposed scheme outperforms state-of-the-art JPEG-XT and 3D-HEVC range coding standards.

Paper Structure

This paper contains 9 sections, 13 equations, 5 figures, 19 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Multi-exposure Stereo Coding Scheme
  4. Color Space Transform
  5. Tucker ALS Low Rank Approximation
  6. Initialize Tucker-ALS algorithm
  7. 3D HEVC Encoding
  8. Experimental Analysis & Results
  9. Conclusion

Figures (5)

  • Figure 1: Workflow of proposed coding scheme for multi-exposure stereo images and efficient HDR compression.
  • Figure 2: Some test multi-exposures stereo images. Note five exposure levels are considered for experiments with every scene.
  • Figure 3: Comparative analysis of proposed scheme at varying tensor rank approximation with 3D-HEVC (IPT Color Space).
  • Figure 4: Comparative analysis of proposed scheme at varying tensor rank approximation with 3D-HEVC ($Y^{'}C_bC_r$ Color Space).
  • Figure 5: Analysis of JPEG XT for coding stereo HDR images.