Exemplar Diffusion: Improving Medical Object Detection with Opportunistic Labels

Abstract

We present a framework to take advantage of existing labels at inference, called \textit{exemplars}, in order to improve the performance of object detection in medical images. The method, \textit{exemplar diffusion}, leverages existing diffusion methods for object detection to enable a training-free approach to adding information of known bounding boxes at test time. We demonstrate that for medical image datasets with clear spatial structure, the method yields an across-the-board increase in average precision and recall, and a robustness to exemplar quality, enabling non-expert annotation. Moreover, we demonstrate how our method may also be used to quantify predictive uncertainty in diffusion detection methods. Source code and data splits openly available online: https://github.com/waahlstrand/ExemplarDiffusion

Figures (3)

• Figure 1: Model overview. Diffusion of boxes $\boldsymbol{x}$ through a forward process $q$ and denoising process $p_\theta$. Known exemplars $\boldsymbol{x}_0^\star$ are replicated $n^\star$ times, forward diffused $\tau$ steps and inserted into $\boldsymbol{x}_T$. The joint set of proposals is denoised normally by the backward process.
• Figure 2: Impact of number of exemplars. Average recall increases with number of exemplars $N^\star$, while average precision decreases beyond $N^\star=2$, on the DENTEX dataset, where the number of labels is approx. 3.5.
• Figure 3: Results visualization. Sample image of teeth detected as diseased by our model (yellow) using a known exemplar (red, dashed). Ellipses indicate an estimated $95\%$ confidence interval for the box coordinates.