Convolutional Neural Network Model Observers Discount Signal-like Anatomical Structures During Search in Virtual Digital Breast Tomosynthesis Phantoms

TL;DR

CNNs are shown to be a more suitable model observer for search tasks like radiologists but not traditional LMOs, which means they can discount false positives arising from anatomical backgrounds.

Abstract

Model observers are computational tools to evaluate and optimize task-based medical image quality. Linear model observers, such as the Channelized Hotelling Observer (CHO), predict human accuracy in detection tasks with a few possible signal locations in clinical phantoms or real anatomic backgrounds. In recent years, Convolutional Neural Networks (CNNs) have been proposed as a new type of model observer. What is not well understood is what CNNs add over the more common linear model observer approaches. We compare the CHO and CNN detection accuracy to the radiologist's accuracy in searching for two types of signals (mass and microcalcification) embedded in 2D/3D breast tomosynthesis phantoms (DBT). We show that the CHO model's accuracy is comparable to the CNN's performance for a location-known-exactly detection task. However, for the search task with 2D/3D DBT phantoms, the CHO's detection accuracy was significantly lower than the CNN accuracy. A comparison to the radiologist's accuracy showed that the CNN but not the CHO could match or exceed the radiologist's accuracy in the 2D microcalcification and 3D mass search conditions. An analysis of the eye position showed that radiologists fixated more often and longer at the locations corresponding to CNN false positives. Most CHO false positives were the phantom's normal anatomy and were not fixated by radiologists. In conclusion, we show that CNNs can be used as an anthropomorphic model observer for the search task for which traditional linear model observers fail due to their inability to discount false positives arising from the anatomical backgrounds.

Figures (8)

• Figure 1: Radiologist study: Twelve radiologists participated in the study. Each was shown a total of 28 2D phantoms and 28 3D phantoms. Half of each set had a signal present and another half with signal absent. Half of the signal-present images contained a microcalcification signal, while the other half had a mass signal. After making a decision, radiologists used a 4-point decision confidence scale where 4 corresponds to strong confidence of signal presence, 3 corresponds to moderate confidence of signal presence, 2 corresponds to moderate confidence of signal absence, and 1 corresponds to strong confidence of signal absence. For the 2D DBT, only eye movements were made, and for 3D, eye movements and scrolling across slices were possible.
• Figure 2: CNN model: Four different CNN models are trained for two modalities (2D/3D) and two signal types (microcalcification/mass). During the test phase, the segmentation-based CNN produces a probability map with a probability value assigned to each pixel, representing its probability of being the signal pixel. A per-pixel threshold computed using the validation set is applied to the CNN output to convert the probability map into a binary map. Connected components are computed using 8 and 26 connectivity for 2D and 3D, respectively, on the resultant binary map.
• Figure 3: 2D search with CHO and FCO: During the testing phase, the 2D template is padded to the size of the 2D input. After taking the Fast Fourier Transform (FFT) of both the input and template, the convolution of the two is performed in the Fourier domain by performing their multiplication. The final response map is generated by taking the inverse FFT of the convolution output in the frequency domain.
• Figure 4: Linear model observers vs. CNN in LKE task: Accuracy (area under the ROC, AUC) for models for the microcalcification (CALC) and mass signals. CNN: Convolutional Neural Network, CHO: Channelized Hotelling Observer; FCO: Filtered Channel Observer
• Figure 5: CNN, linear model observes (CHO, FCO) and radiologists in 2D and 3D search: For CALC signal, radiologists underperformed in 3D search, whereas model observers improved their performance from 2D to 3D. For MASS, where the CNN performance still increased from 2D to 3D, CHO and FCO underperformed in 3D, where a better integration across slices is needed.