Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Polyp Segmentation from an Out-of-Distribution Perspective

Ge-Peng Ji, Jing Zhang, Dylan Campbell, Huan Xiong, Nick Barnes

TL;DR

The work addresses automatic colorectal polyp segmentation under limited labeled data by reframing it as an out-of-distribution (OOD) detection problem. It trains a masked autoencoder (MAE) on healthy colonoscopy images to learn in-distribution representations and uses reconstruction differences to generate per-pixel anomaly scores for polyp detection, aided by inference-time latent-space standardisation. The authors demonstrate strong in-domain performance and notable generalisation to multiple out-of-domain datasets, with ablations confirming the importance of masking ratio and latent-space standardisation. This approach offers a simple, self-supervised pathway to robust polyp segmentation with broad cross-dataset applicability.

Abstract

Unlike existing fully-supervised approaches, we rethink colorectal polyp segmentation from an out-of-distribution perspective with a simple but effective self-supervised learning approach. We leverage the ability of masked autoencoders -- self-supervised vision transformers trained on a reconstruction task -- to learn in-distribution representations; here, the distribution of healthy colon images. We then perform out-of-distribution reconstruction and inference, with feature space standardisation to align the latent distribution of the diverse abnormal samples with the statistics of the healthy samples. We generate per-pixel anomaly scores for each image by calculating the difference between the input and reconstructed images and use this signal for out-of-distribution (ie, polyp) segmentation. Experimental results on six benchmarks show that our model has excellent segmentation performance and generalises across datasets. Our code is publicly available at https://github.com/GewelsJI/Polyp-OOD.

Rethinking Polyp Segmentation from an Out-of-Distribution Perspective

TL;DR

The work addresses automatic colorectal polyp segmentation under limited labeled data by reframing it as an out-of-distribution (OOD) detection problem. It trains a masked autoencoder (MAE) on healthy colonoscopy images to learn in-distribution representations and uses reconstruction differences to generate per-pixel anomaly scores for polyp detection, aided by inference-time latent-space standardisation. The authors demonstrate strong in-domain performance and notable generalisation to multiple out-of-domain datasets, with ablations confirming the importance of masking ratio and latent-space standardisation. This approach offers a simple, self-supervised pathway to robust polyp segmentation with broad cross-dataset applicability.

Abstract

Unlike existing fully-supervised approaches, we rethink colorectal polyp segmentation from an out-of-distribution perspective with a simple but effective self-supervised learning approach. We leverage the ability of masked autoencoders -- self-supervised vision transformers trained on a reconstruction task -- to learn in-distribution representations; here, the distribution of healthy colon images. We then perform out-of-distribution reconstruction and inference, with feature space standardisation to align the latent distribution of the diverse abnormal samples with the statistics of the healthy samples. We generate per-pixel anomaly scores for each image by calculating the difference between the input and reconstructed images and use this signal for out-of-distribution (ie, polyp) segmentation. Experimental results on six benchmarks show that our model has excellent segmentation performance and generalises across datasets. Our code is publicly available at https://github.com/GewelsJI/Polyp-OOD.
Paper Structure (13 sections, 4 equations, 3 figures, 2 tables)

This paper contains 13 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminaries: the Masked Autoencoder (MAE)
  4. In-distribution Training Pipeline
  5. Out-of-distribution Inference Pipeline
  6. Experiment
  7. Experiment Setup
  8. In-domain Performance
  9. Out-of-domain (Generalisation) Performance
  10. Qualitative Visualisation
  11. Diagnostic Experiment
  12. Discussion
  13. Conclusion

Figures (3)

  • Figure 1: Illustration of our Polyp-OOD model. (a) ID training: the model is trained to reconstruct the masked-out patches from a healthy sample $\mathbf{X}_{\text{id}}$. (b) OOD inference: we generate an anomaly score $\mathbf{A}$ by calculating the reconstruction difference between abnormal input $\mathbf{X}_{\text{ood}}$ and its reconstruction $\mathbf{R}_{\text{ood}}$.
  • Figure 2: The qualitative results of our Polyp-OOD on in-distribution (left part) and out-of-distribution (right part) samples.
  • Figure 3: The effect of different masking ratios, including in-domain (Left: SUN-SEG-I (Hard) ji2022video) and out-of-domain (Right: CVC-ColonDB bernal2012towards) datasets.