A Versatile Depth Video Encoding Scheme Based on Low-rank Tensor Modeling for Free Viewpoint Video
Mansi Sharma, Jyotsana Grover
TL;DR
This work targets efficient depth map compression for free-viewpoint video by modeling depth sequences as a high-order tensor and applying CANDECOMP/PARAFAC (CP) decomposition with rank $R$. The depth tensor $ abla T_{MES}^{c}$ is approximated by CP factor matrices $[[ extbf{S}^{(1)}, \, \ldots, extbf{S}^{(N)}]]$, enabling a low-rank representation; the factor matrices are then encoded using HEVC intra coding with adjustable quantization to achieve multiple bitrates. The CP-ALS optimization is accelerated by pairwise perturbation, reducing complexity while preserving multi-dimensional structure. The decoded depth maps are used for view synthesis via DIBR, and experiments on Ballet and Breakdancing sequences demonstrate substantial rate reductions (BDBR) with maintained PSNR/SSIM of the synthesized views, indicating scalable depth coding suitable for multi-view displays and baseline adaptation across devices.
Abstract
The compression quality losses of depth sequences determine quality of view synthesis in free-viewpoint video. The depth map intra prediction in 3D extensions of the HEVC applies intra modes with auxiliary depth modeling modes (DMMs) to better preserve depth edges and handle motion discontinuities. Although such modes enable high efficiency compression, but at the cost of very high encoding complexity. Skipping conventional intra coding modes and DMMs in depth coding limits practical applicability of the HEVC for 3D display applications. In this paper, we introduce a novel low-complexity scheme for depth video compression based on low-rank tensor decomposition and HEVC intra coding. The proposed scheme leverages spatial and temporal redundancy by compactly representing the depth sequence as a high-order tensor. Tensor factorization into a set of factor matrices following CANDECOMP PARAFAC (CP) decomposition via alternating least squares give a low-rank approximation of the scene geometry. Further, compression of factor matrices with HEVC intra prediction support arbitrary target accuracy by flexible adjustment of bitrate, varying tensor decomposition ranks and quantization parameters. The results demonstrate proposed approach achieves significant rate gains by efficiently compressing depth planes in low-rank approximated representation. The proposed algorithm is applied to encode depth maps of benchmark Ballet and Breakdancing sequences. The decoded depth sequences are used for view synthesis in a multi-view video system, maintaining appropriate rendering quality.