Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring

TL;DR

This work addresses the challenge of detecting out-of-distribution plankton images under real-world distribution shifts. It introduces a four-way OoD benchmark based on DYB-PlanktonNet and systematically evaluates 22 post hoc detectors using the OpenOOD framework, across a model zoo of backbones. Distance-based methods often excel in Far-OoD scenarios, while ViM provides the most robust overall performance across both Far- and Near-OoD benchmarks. The study delivers a practical reference for deploying plankton recognition systems and highlights directions for future research, including fine-grained and multi-label OoD detection. The accompanying code base enables researchers to reproduce and extend the benchmark results.

Abstract

Automated plankton recognition models face significant challenges during real-world deployment due to distribution shifts (Out-of-Distribution, OoD) between training and test data. This stems from plankton's complex morphologies, vast species diversity, and the continuous discovery of novel species, which leads to unpredictable errors during inference. Despite rapid advancements in OoD detection methods in recent years, the field of plankton recognition still lacks a systematic integration of the latest computer vision developments and a unified benchmark for large-scale evaluation. To address this, this paper meticulously designed a series of OoD benchmarks simulating various distribution shift scenarios based on the DYB-PlanktonNet dataset \cite{875n-f104-21}, and systematically evaluated twenty-two OoD detection methods. Extensive experimental results demonstrate that the ViM \cite{wang2022vim} method significantly outperforms other approaches in our constructed benchmarks, particularly excelling in Far-OoD scenarios with substantial improvements in key metrics. This comprehensive evaluation not only provides a reliable reference for algorithm selection in automated plankton recognition but also lays a solid foundation for future research in plankton OoD detection. To our knowledge, this study marks the first large-scale, systematic evaluation and analysis of Out-of-Distribution data detection methods in plankton recognition. Code is available at https://github.com/BlackJack0083/PlanktonOoD.

Figures (5)

• Figure 1: Our constructed plankton Out-of-Distribution detection image benchmark comprises four distinct distribution shift scenarios: ID, Near-OoD, Far-OoD (Bubbles & Particles), and Far-OoD (General). For each distribution, we provide representative class images. A detailed classification can be found in the Supplementary Material.
• Figure 2: Distance-based Methods. The solid points on the line graph represent the average values, with the standard deviation range illustrated by the shaded area between the dashed lines.
• Figure 3: Classification-based Methods. The solid points on the line graph represent the average values, with the standard deviation range illustrated by the shaded area between the dashed lines.
• Figure 4: Density-based Methods. The solid points on the line graph represent the average values, with the standard deviation range illustrated by the shaded area between the dashed lines.
• Figure 5: Correlation Between ID Classification Accuracy and OoD Detection Performance. We selected five representative methods: MSP, ViM, Energy, KNN, and Mahalanobis, then we evaluated these methods using four common network architectures: ResNet-18, ResNet-50, DenseNet-121, and ViT, on our Near-OoD, Far-OoD (Bubbles & Particles), and Far-OoD (General) benchmarks. The average performance of these methods across different architectures was plotted on scatter graphs to visually analyze their correlation.