Fast Multirate Encoding for 360° Video in OMAF Streaming Workflows

TL;DR

This work tackles the prohibitive encoding time required to generate full multirepresentation ladders for 360° HAS streams in OMAF, especially for 8K ERP content. It introduces projection-aware fast multirate encoding with cross-resolution and cross-QP analysis reuse, implemented via two sharing schemes (CRC and PRA) and extended to cubemap tiling (CMP) to exploit parallelism. Evaluations on the SJTU 8K 360° dataset show substantial encoding-time reductions (roughly 33–59% for ERP and about 51% on average for CMP) with minimal rate-distortion degradation, and wall-clock speedups up to 4.2x. The proposed framework integrates with standard OMAF/DASH pipelines and offers practical benefits for large VR libraries and potentially live VR streaming, with future work on VVC, ML-based anchor selection, and viewport QoE integration.

Abstract

Preparing high-quality 360-degree video for HTTP Adaptive Streaming requires encoding each sequence into multiple representations spanning different resolutions and quantization parameters (QPs). For ultra-high-resolution immersive content such as 8K 360-degree video, this process is computationally intensive due to the large number of representations and the high complexity of modern codecs. This paper investigates fast multirate encoding strategies that reduce encoding time by reusing encoder analysis information across QPs and resolutions. We evaluate two cross-resolution information-reuse pipelines that differ in how reference encodes propagate across resolutions: (i) a strict HD -> 4K -> 8K cascade with scaled analysis reuse, and (ii) a resolution-anchored scheme that initializes each resolution with its own highest-bitrate reference before guiding dependent encodes. In addition to evaluating these pipelines on standard equirectangular projection content, we also apply the same two pipelines to cubemap-projection (CMP) tiling, where each 360-degree frame is partitioned into independently encoded tiles. CMP introduces substantial parallelism, while still benefiting from the proposed multirate analysis-reuse strategies. Experimental results using the SJTU 8K 360-degree dataset show that hierarchical analysis reuse significantly accelerates HEVC encoding with minimal rate-distortion impact across both equirectangular and CMP-tiled content, yielding encoding-time reductions of roughly 33%-59% for ERP and about 51% on average for CMP, with Bjontegaard Delta Encoding Time (BDET) gains approaching -50% and wall-clock speedups of up to 4.2x.

Figures (6)

• Figure 1: Rate-quality, and rate-encoding time curves of Kaixuanmenye panoliu_sjtu_2017. Here, quality is represented using WSPSNR.
• Figure 2: Comparison of 360° Projection Formats: ERP versus CMP.
• Figure 3: Overview of the proposed fast multirate encoding pipeline. The optional ERP$\rightarrow$CMP conversion is applied only in CMP-CRC and CMP-PRA . For ERP-CRC and ERP-PRA , the pipeline omits the projection box; the multirate reuse logic (anchor + dependents, analysis save/load) remains identical.
• Figure 4: Complexity analysis of the SJTU dataset using $(E_Y, h)$ from VCA vca_ref. Higher values indicate more complex motion and texture.
• Figure 5: Quality evaluation framework.