Which images to label for few-shot medical landmark detection?

TL;DR

The paper tackles the challenge of label scarcity in medical landmark detection by focusing on selecting the most informative templates for annotation. It proposes Sample Choosing Policy (SCP), a three-stage framework that (i) learns dense features via self-supervised pixel-wise contrastive learning, (ii) uses SIFT-based key-point proposals as substitutes for landmarks, and (iii) computes a representative similarity score to select the most informative templates. Empirical results show substantial reductions in mean radial error (MRE) on Cephalometric Xray ($3.595\to3.083$ mm) and Hand Xray ($4.114\to2.635$ mm) in one-shot settings, with further gains after refinement and in facial landmark tasks like WFLW. The work demonstrates that selecting representative templates, rather than only focusing on hard examples, can significantly improve few-shot medical landmark detection while reducing annotation effort.

Abstract

The success of deep learning methods relies on the availability of well-labeled large-scale datasets. However, for medical images, annotating such abundant training data often requires experienced radiologists and consumes their limited time. Few-shot learning is developed to alleviate this burden, which achieves competitive performances with only several labeled data. However, a crucial yet previously overlooked problem in few-shot learning is about the selection of template images for annotation before learning, which affects the final performance. We herein propose a novel Sample Choosing Policy (SCP) to select "the most worthy" images for annotation, in the context of few-shot medical landmark detection. SCP consists of three parts: 1) Self-supervised training for building a pre-trained deep model to extract features from radiological images, 2) Key Point Proposal for localizing informative patches, and 3) Representative Score Estimation for searching the most representative samples or templates. The advantage of SCP is demonstrated by various experiments on three widely-used public datasets. For one-shot medical landmark detection, its use reduces the mean radial errors on Cephalometric and HandXray datasets by 14.2% (from 3.595mm to 3.083mm) and 35.5% (4.114mm to 2.653mm), respectively.

Figures (6)

• Figure 1: The distribution of the mean radial error (MRE) when choosing a different image as a template in one-shot medical landmark detection task. The x-axis refers to MRE and the y-axis refers to the percentage of MRE lying in the corresponding ranges. Evidently, the choice of template affects the performance significantly.
• Figure 2: Difference from active learning. Deep models can remember and cluster the images or patterns they viewed. Instead of active learning (AL) tending to find the unfamiliar examples, our goal is to find the ones nearest to the center of latent space which we think more representative and important when only several images can be labeled.
• Figure 3: Overview of Sample Choosing Policy (SCP): SCP consists of three stages. Stage 1: Extract features $f_i$ from image $X_i$ via a self-trained deep model. Stage 2: Detect key points $Q^{X_i}$ by traditional method like SIFT from image $X_i$. Stage 3: Evaluate similarities between filtered features $\hat{f}_i$ and features $g$ of templates $\{T_m\}$, and record the average similarity $R[{\{T_m\}}]$. The combination of templates with highest similarity $\hat{R}$ are selected as $\{\hat{T}_m\}$.
• Figure 4: Visual Comparison of templates from our policy and random selection. Column "Template/Test 1/Test 2" refers to the templates and two test images. The row "Ours" and "Random" refers to the template selected by our method and random selection, respectively. As shown in red dashed boxes, our template outperforms the random selected template in visualization.
• Figure 5: Similarities of potential key points vs. landmarks. The correlation coefficient (CC) of potential key points and landmarks is 0.462, thus we think it is feasible to replace landmarks with potential key points when estimating similarities.