Content-Distortion High-Order Interaction for Blind Image Quality Assessment

TL;DR

This work tackles the challenge of NR-IQA by addressing the complex interactions between image content and distortions. It introduces CoDI-IQA, which uses separate Content-Aware and Distortion-Aware Encoders and a Progressive Perception Interaction Module to realize high-order, hierarchical interactions across multiple feature levels. The approach demonstrates strong data efficiency and generalization across diverse synthetic and authentic datasets, outperforming state-of-the-art methods in many settings. The results suggest that explicitly modeling content–distortion interactions in a hierarchical, interaction-guided framework can yield more reliable and robust image quality predictions with limited training data.

Abstract

The content and distortion are widely recognized as the two primary factors affecting the visual quality of an image. While existing No-Reference Image Quality Assessment (NR-IQA) methods have modeled these factors, they fail to capture the complex interactions between content and distortions. This shortfall impairs their ability to accurately perceive quality. To confront this, we analyze the key properties required for interaction modeling and propose a robust NR-IQA approach termed CoDI-IQA (Content-Distortion high-order Interaction for NR-IQA), which aggregates local distortion and global content features within a hierarchical interaction framework. Specifically, a Progressive Perception Interaction Module (PPIM) is proposed to explicitly simulate how content and distortions independently and jointly influence image quality. By integrating internal interaction, coarse interaction, and fine interaction, it achieves high-order interaction modeling that allows the model to properly represent the underlying interaction patterns. To ensure sufficient interaction, multiple PPIMs are employed to hierarchically fuse multi-level content and distortion features at different granularities. We also tailor a training strategy suited for CoDI-IQA to maintain interaction stability. Extensive experiments demonstrate that the proposed method notably outperforms the state-of-the-art methods in terms of prediction accuracy, data efficiency, and generalization ability.

Figures (9)

• Figure 1: Image on top: performance of the proposed CoDI-IQA with varying amounts of training data on the KonIQ-10K KonIQ dataset. The state-of-the-art (SOTA) results are obtained from LoDa xu2024LoDa with 80% data, whereas CoDI-IQA can outperforms it using only 30% data. Image at bottom: Ours CoDI-IQA compared with several SOTA models, showing exceptional improvements in cross-dataset settings on real-world images. The evaluation metric used here is SRCC. Ours (R) and Ours (S) denote CoDI-IQA using ResNet50 he2016resnet and Swin-Base Transformer liu2021swin as the CAE, respectively.
• Figure 2: Images in the first column: the distorted images in the KonIQ-10K KonIQ dataset. Images in the second column: the attention maps from the CAE. Images in the third column: the attention maps from the DAE. Images in the last column: the 3D visualizations derived from columns two and three.
• Figure 3: Comparison between existing methods and the proposed method for interaction modeling in NR-IQA. Representatives include: (a) DBCNN zhang2020DBCNN; (b) Su et al.su2023DisManifold and Re-IQA saha2023ReIQA; (c) CDINet zheng2024CDINet; and (d) our PPIM, which is compatible with the interaction properties. More details of (d) can be found in Fig. \ref{['fig4']}. Feature maps with red glow correspond to distortion features, whereas those with green glow represent to content features.
• Figure 4: The proposed CoDI-IQA involves the CAE and DAE for feature extraction, the PPIM for high-order interaction, and a quality prediction module for generating quality scores.
• Figure 5: The architecture of PPIM. The detailed flowchart outlines the processes involved in high-order interaction.