Multitask Auxiliary Network for Perceptual Quality Assessment of Non-Uniformly Distorted Omnidirectional Images

TL;DR

This work tackles non-uniform distortion in omnidirectional image quality assessment by introducing MTAOIQA, a multitask auxiliary network that jointly learns a main quality prediction task with distortion-range, distortion-type, and distortion-degree auxiliary tasks. The model employs a Swin Transformer backbone to extract multiscale viewport features, a Multitask Feature Selection module to allocate features to tasks, and a Multitask Auxiliary Fusion mechanism to integrate information across tasks. Extensive experiments on JUFE-10K and OIQ-10K demonstrate state-of-the-art performance and show that auxiliary tasks and the designed modules substantially improve robustness to non-uniform distortions. The approach offers a practical framework for improved QoE-driven VR/omnidirectional image processing, with publicly available code to facilitate adoption and further research.

Abstract

Omnidirectional image quality assessment (OIQA) has been widely investigated in the past few years and achieved much success. However, most of existing studies are dedicated to solve the uniform distortion problem in OIQA, which has a natural gap with the non-uniform distortion problem, and their ability in capturing non-uniform distortion is far from satisfactory. To narrow this gap, in this paper, we propose a multitask auxiliary network for non-uniformly distorted omnidirectional images, where the parameters are optimized by jointly training the main task and other auxiliary tasks. The proposed network mainly consists of three parts: a backbone for extracting multiscale features from the viewport sequence, a multitask feature selection module for dynamically allocating specific features to different tasks, and auxiliary sub-networks for guiding the proposed model to capture local distortion and global quality change. Extensive experiments conducted on two large-scale OIQA databases demonstrate that the proposed model outperforms other state-of-the-art OIQA metrics, and these auxiliary sub-networks contribute to improve the performance of the proposed model. The source code is available at https://github.com/RJL2000/MTAOIQA.

Figures (7)

• Figure 1: An intuitive comparison between the proposed multitask auxiliary OIQA model, single task OIQA framework, and existing multitask OIQA framework.
• Figure 2: The framework of the proposed model. It consists of three optional auxiliary tasks and a main task, where three auxiliary tasks are distortion range prediction, distortion type classification, and distortion degree classification, respectively, and the main task is quality score prediction. To connect these tasks seamlessly, a MFS module is used to adaptively select specific features for different tasks, and a MAF module is used to combine the features from different tasks for global quality prediction.
• Figure 3: The multitask feature selection module.
• Figure 4: The architecture of the core modules in three optional auxiliary networks.
• Figure 5: The multidimension distortion integration module.