Observer-Usable Information as a Task-specific Image Quality Metric

TL;DR

This work addresses the limitation of ideal-observer metrics by introducing predictive V-information, an observer-aware, information-theoretic measure for task-based IQ. It defines I_V[X -> Y] as the difference between the total task entropy and the best conditioned entropy achievable by a restricted predictor family, enabling assessment under sub-ideal observers and across multi-class tasks. Through stylized low-field MRI restoration experiments, V-information correlates with AUC for binary tasks but remains informative beyond saturation and extends naturally to multi-class problems. The framework offers a scalable, observer-specific IQ metric that complements traditional SDT measures and supports stable system-level comparisons across imaging pipelines and observer types.

Abstract

Objective, task-based measures of image quality (IQ) have been widely advocated for assessing and optimizing medical imaging technologies. Besides signal detection theory-based measures, information-theoretic quantities have been proposed to quantify task-based IQ. For example, task-specific information (TSI), defined as the mutual information between an image and a task variable, represents an optimal measure of how informative an image is for performing a specified task. However, like the ideal observer from signal detection theory, TSI does not quantify the amount of task-relevant information in an image that can be exploited by a sub-ideal observer. A recently proposed relaxation of TSI, termed predictive V-information (V-info), removes this limitation and can quantify the utility of an image with consideration of a specified family of sub-ideal observers. In this study, for the first time, we introduce and investigate V-info as an objective, task-specific IQ metric. To corroborate its usefulness, a stylized magnetic resonance image restoration problem is considered in which V-info is employed to quantify signal detection or discrimination performance. The presented results show that V-info correlates with area under the receiver operating characteristic (ROC) curve for binary tasks, while being readily applicable to multi-class (>2) tasks where ROC analysis is challenging. Notably, V-info exhibits greater sensitivity in scenarios where conventional metrics saturate. These findings demonstrate that V-info represents a new objective IQ measure that can complement conventional signal detection theory-based ones.

Figures (12)

• Figure 1: MRI restoration network architecture. The dimensions of the input and output images are 288 $\times$ 320.
• Figure 2: Representative examples of high-field, low-field, and restored high-field MRI images are shown. Columns correspond to (a) high-field MRI, (b) low-field MRI, and (c) restored high-field MRI. The red box highlights the signal. The first and second rows show one-signal-present images with background noise levels of 35 and 45, respectively, and a signal amplitude of 0.1. The third and fourth rows show two-signal-present images with noise levels of 35 and 45 and signal amplitude of 0.15, respectively.
• Figure 3: Observer performance on the binary signal detection task using a balanced dataset as quantified by $\mathcal{V}$-info (a) and AUC (b), shown as functions of the number of CNN NO modules in the observer. Both metrics exhibit similar overall trends, indicating improved performance with increasing observer capacity across low-field, restored, and high-field image types. However, as shown in the first and third rows, while AUC saturates at higher capacities and fails to distinguish further performance gains, $\mathcal{V}$-info continues to increase, providing a more sensitive measure of observer performance in these regimes.
• Figure 4: Relationship between $\mathcal{V}$-info and AUC for CNN-based observers on the binary signal-detection task using a balanced dataset. Blue dots denote observer performance achieved with five capacity levels (2 – 6 CNN NO modules) under each imaging condition (low-field, high-field, restored). A clear linear dependence is observed, with the coefficient of determination $R^{2}$ approaching 1.
• Figure 5: Comparison of AUC and $\mathcal{V}$-info for a binary classification task. Blue and orange curves represent predicted probability distributions for signal-absent and signal-present classes, respectively. Although AUC values appear similar across the two examples, $\mathcal{V}$-info highlights a larger difference between the underlying probability distributions, illustrating its higher sensitivity.