Likelihood ratio for a binary Bayesian classifier under a noise-exclusion model

TL;DR

This work addresses binary diagnostic decisions when data are partially available due to feature-exclusion by introducing a noise-exclusion, gist-processing ideal observer. It develops a truncated data framework where thresholds $\bm{\tau}$ gate external feature distributions and combine with internal noise to form a mixture ROC, with $\lambda_{\alpha,\tau}$ and a three-component decomposition of AUC into analysis, gist, and guessing contributions. Analytic results and simulations show that, under substantial internal noise, truncation can improve ROC performance and that optimal thresholds tend to lie near class means; these findings have implications for medical imaging trial design, computer-vision benchmarking, and sensor evaluation where data reduction is common. The framework provides a principled way to quantify when data truncation helps or hurts diagnostic performance, guiding the design of thresholds and processing pipelines in noisy, incomplete-information settings.

Abstract

We develop a new statistical ideal observer model that performs holistic visual search (or gist) processing in part by placing thresholds on minimum extractable image features. In this model, the ideal observer reduces the number of free parameters thereby shrinking down the system. The applications of this novel framework is in medical image perception (for optimizing imaging systems and algorithms), computer vision, benchmarking performance and enabling feature selection/evaluations. Other applications are in target detection and recognition in defense/security as well as evaluating sensors and detectors.

Figures (4)

• Figure 1: Handling unrated ROC cases with the proposed truncation analysis. (a) With lefthand truncation, the partial curve (shown as the solid line) results in the reduced area $A_{z}^{(1)}$ represented by the shaded region. Assigning the minimum rating ($\lambda$ = $-\infty$) to all unrated negative cases provides an additional area $A_{z}^{(2)}$ by completing the curve with the horizontal dashed line. This completion disallows credit for guessing, whereas also assigning the minimum rating to unrated positive cases produces the dotted-line guessing extension with area increment $A_{z}^{(3)}$. (b) ROC curve completion based on righthand truncation. The ROC curves were computed using a single feature.
• Figure 2: ROC curves from the ideal and truncated ideal observers. A single feature was considered. The dotted and solid lines represent the ideal observer respectively with and without internal noise of variance $\sigma^{2}$. The dashed line is from the thresholded model with the same internal-noise variance.
• Figure 3: Comparison of truncation model performance for three single-feature simulations. In these simulations, the normal feature followed the standard distribution, while the abnormal feature mean was 0.75. From left to right, the standard deviation for the abnormal feature was 0.25, 1.0 and 3.0. The top row (plots a-c) shows overall performance (in black) without internal noise. Also shown are the analysis (red), gist (blue) and guessing (green) contributions. The bottom row (plots d-f) displays overall performance as a function of threshold for internal noise $\sigma$$\in$$\{0,8,\infty\}$.
• Figure 4: Results of two-feature trial with x mark showing maximum of ideal observer performance for a given noise variance (shown by sigma) as the data is truncated. The x and y axis shows the range of values for each variable respectively with 0 representing the mid point of the distribution. External noise is shown by $\sigma1$ with normal distribution given by $\sigma1$=1