Benchmarking Image Transformers for Prostate Cancer Detection from Ultrasound Data

TL;DR

Prostate cancer detection in micro-ultrasound often relies on ROI-scale CNNs with weak ROI labels; this paper benchmarks Vision Transformers for ROI-scale and multi-scale classification, complemented by a novel multi-objective learning strategy that fuses ROI and core predictions. Using VICReg self-supervised pretraining on 693 patients across 5 centers, the study trains ViT, CCT, PvT backbones and compares to ResNet18 baselines, applying a BERT classifier for core prediction in MIL. Results show CNNs outperform transformers on ROI-scale and MIL underperform with certain backbones, but multi-objective learning yields a notable AUROC of 0.779 and improved metrics, especially with ResNet18+BERT MO. The findings suggest transformers are less suitable for this small dataset, while multi-scale MIL with MO learning offers a practical improvement and points to dataset size as a key factor for transformer efficacy.

Abstract

PURPOSE: Deep learning methods for classifying prostate cancer (PCa) in ultrasound images typically employ convolutional networks (CNNs) to detect cancer in small regions of interest (ROI) along a needle trace region. However, this approach suffers from weak labelling, since the ground-truth histopathology labels do not describe the properties of individual ROIs. Recently, multi-scale approaches have sought to mitigate this issue by combining the context awareness of transformers with a CNN feature extractor to detect cancer from multiple ROIs using multiple-instance learning (MIL). In this work, we present a detailed study of several image transformer architectures for both ROI-scale and multi-scale classification, and a comparison of the performance of CNNs and transformers for ultrasound-based prostate cancer classification. We also design a novel multi-objective learning strategy that combines both ROI and core predictions to further mitigate label noise. METHODS: We evaluate 3 image transformers on ROI-scale cancer classification, then use the strongest model to tune a multi-scale classifier with MIL. We train our MIL models using our novel multi-objective learning strategy and compare our results to existing baselines. RESULTS: We find that for both ROI-scale and multi-scale PCa detection, image transformer backbones lag behind their CNN counterparts. This deficit in performance is even more noticeable for larger models. When using multi-objective learning, we can improve performance of MIL, with a 77.9% AUROC, a sensitivity of 75.9%, and a specificity of 66.3%. CONCLUSION: Convolutional networks are better suited for modelling sparse datasets of prostate ultrasounds, producing more robust features than transformers in PCa detection. Multi-scale methods remain the best architecture for this task, with multi-objective learning presenting an effective way to improve performance.

Figures (1)

• Figure 1: Multi-scale classification of prostate cancer across the whole biopsy core using BERT and multi-objective learning.