LFACon: Introducing Anglewise Attention to No-Reference Quality Assessment in Light Field Space

TL;DR

The paper tackles the problem of no-reference light field image quality assessment (LFIQA), where both spatial quality and angular consistency across subviews must be evaluated efficiently. It introduces anglewise attention, comprising three kernels—anglewise self-attention, anglewise grid attention, and anglewise central attention—within a lightweight framework (LF-DSC) to form the LFACon metric. Empirical results on Win5-LID, SMART, and MPI-LFA show that LFACon achieves state-of-the-art NR-LFIQA performance with notable gains in RMSE, SRCC, and PLCC and significantly lower computation time. The work provides a strong foundation for practical LFIQA and suggests extensions to other light-field tasks and 3D representations.

Abstract

Light field imaging can capture both the intensity information and the direction information of light rays. It naturally enables a six-degrees-of-freedom viewing experience and deep user engagement in virtual reality. Compared to 2D image assessment, light field image quality assessment (LFIQA) needs to consider not only the image quality in the spatial domain but also the quality consistency in the angular domain. However, there is a lack of metrics to effectively reflect the angular consistency and thus the angular quality of a light field image (LFI). Furthermore, the existing LFIQA metrics suffer from high computational costs due to the excessive data volume of LFIs. In this paper, we propose a novel concept of "anglewise attention" by introducing a multihead self-attention mechanism to the angular domain of an LFI. This mechanism better reflects the LFI quality. In particular, we propose three new attention kernels, including anglewise self-attention, anglewise grid attention, and anglewise central attention. These attention kernels can realize angular self-attention, extract multiangled features globally or selectively, and reduce the computational cost of feature extraction. By effectively incorporating the proposed kernels, we further propose our light field attentional convolutional neural network (LFACon) as an LFIQA metric. Our experimental results show that the proposed LFACon metric significantly outperforms the state-of-the-art LFIQA metrics. For the majority of distortion types, LFACon attains the best performance with lower complexity and less computational time.

Figures (9)

• Figure 1: Different attention models for LFIs: (a) The existing LFI attention kernels directly multiply an LFI by a learned attention map. (b) The proposed kernels introduce learnable keys, queries, and values to better model the self-attention in various patterns (marked in different colors).
• Figure 2: Angular domain coverage in the attention computation: (a) The EPI-based kernels process the target subview by only utilizing partial source subviews (depending on the relative epipolar geometry). (b) The three proposed kernels utilize all the source subviews to compute the target subview's self-attention in three patterns with flexible complexity.
• Figure 3: Structure of the proposed anglewise attention kernels, including anglewise self-attention, anglewise grid attention, and anglewise central attention. The proposed anglewise attention kernels contain powerful multihead self-attention kernels and lightweight convolution operations (in pre- and postattention kernels), which are capsuled into a single generalized kernel for efficient feature extraction.
• Figure 4: Structure of the proposed LFACon metric, which incorporates the proposed anglewise attention kernels.
• Figure 5: Visualization of the proposed anglewise attention: The workflows and attention maps for the incorporated attention kernels in LFACon for three sample LFIs are shown in the three rows. In each row, the first column shows the sample LFI with four representative query locations marked in red, pink, blue, and green. The remaining seven columns show the anglewise attention maps (computed at different stages) for those query locations. The colored locations correspond to the query locations. The other locations are marked with different grayscale levels to visualize the different attention weights.