py360tool: Um framework para manipulação de vídeo 360$^\circ$ com ladrilhos

TL;DR

The paper tackles the high bandwidth demands of 360° video streaming by advocating a tile-based approach that focuses on the viewport. It introduces py360tools, an open-source Python library that automates key client-side tasks—viewport prediction, tile selection, projection reconstruction, and viewport extraction—and supports multiple projections (e.g., Cubemap and Equi-rectangular) to analyze tiling strategies. The framework models the geometry of tiled spherical video and uses a yaw-pitch-roll viewport rotation along with a FOV pyramid defined by four planes to determine visible tiles, enabling QoE/QoS evaluation under various conditions. The work provides a foundation for research into tiling and ABR strategies, with future plans including ABR and viewport-prediction modules and NS-3 integration for more realistic networking simulations.

Abstract

The streaming of 360$^\circ$ videos is one of the most bandwidth-demanding virtual reality (VR) applications, as the video must be encoded in ultra-high resolution to ensure an immersive experience. To optimize its transmission, current approaches partition the spherical video into tiles, which are encoded at different bitrates and selectively delivered, based on the viewing direction of the user (viewport). The complexity of this architecture, which involves viewport prediction, tile selection, bit rate adaptation, and handling of parallel streaming, requires new tools to evaluate quality of experience (QoE) and quality of service (QoS), especially due to its interactive nature and low reproducibility. This work introduces py360tools, a Python library to handle tile-based 360$^\circ$ video streaming. The library automates key client-side tasks, such as spherical projection reconstruction, viewport extraction, and tile selection, facilitating the playback and simulation of streaming sessions. Furthermore, py360tools offers a flexible architecture, enabling efficient analysis of different projections and tiling strategies.

Figures (6)

• Figure 1: Antes de ser transmitido o vídeo é projetado, recortado, comprimido e segmentado em chunks. O cliente solicita apenas os chunks que aparecem no viewport
• Figure 2: Internamente o cliente pode ser dividido em: preditor de viewport, seletor de ladrilhos, ABR, Cliente HTTP, decodificador, montador de ladrilhos e extrator de viewport. Estes módulos operam de forma sequencial e circular durante a reprodução do vídeo
• Figure 3: As transformações ocorrem em duas dimensões. A dimensão da esfera é manipulada em coordenadas cartesianas ou em ângulos de Euler. A dimensão da imagem é onde a codificação é efetuada e as distorções são geradas.
• Figure 4: Diagrama de classes simplificado da py360tools.
• Figure 5: O que é visível (FOV) compreende a região do espaço que faz intersecção com quatro planos concêntricos. Basta um único pixel de um ladrilho estar dentro do FOV para que seja considerado visto.