Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images

Rita Pucci, Vincent J. Kalkman, Dan Stowell

TL;DR

This work conducts a comprehensive, cross-family comparison of nine deep learning architectures (CNN, ViT, LBVT) for fine-grained insect species classification using crowdsourced images from Observation.org and Artportalen. It evaluates four facets—classification performance, embedding quality, computational cost, and gradient activity—through per-dataset and per-species analyses, including knowledge-distillation ablations. The study finds Locality-based Vision Transformers (LBVT, notably ViTAE) delivering the best embedding quality and species-level separation, Vision Transformers (ViT, especially with distillation) offering strong inference efficiency, and CNNs providing robust baselines with a trade-off in embedding and long-range dependency modeling. The results inform practical guidance for biodiversity monitoring tasks, highlighting when to prioritize accuracy, embedding structure, or computational efficiency, and illustrating generalization challenges across data sources like Observation.org and Artportalen.

Abstract

With fine-grained classification, we identify unique characteristics to distinguish among classes of the same super-class. We are focusing on species recognition in Insecta, as they are critical for biodiversity monitoring and at the base of many ecosystems. With citizen science campaigns, billions of images are collected in the wild. Once these are labelled, experts can use them to create distribution maps. However, the labelling process is time-consuming, which is where computer vision comes in. The field of computer vision offers a wide range of algorithms, each with its strengths and weaknesses; how do we identify the algorithm that is in line with our application? To answer this question, we provide a full and detailed evaluation of nine algorithms among deep convolutional networks (CNN), vision transformers (ViT), and locality-based vision transformers (LBVT) on 4 different aspects: classification performance, embedding quality, computational cost, and gradient activity. We offer insights that we haven't yet had in this domain proving to which extent these algorithms solve the fine-grained tasks in Insecta. We found that the ViT performs the best on inference speed and computational cost while the LBVT outperforms the others on performance and embedding quality; the CNN provide a trade-off among the metrics.

Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images

TL;DR

This work conducts a comprehensive, cross-family comparison of nine deep learning architectures (CNN, ViT, LBVT) for fine-grained insect species classification using crowdsourced images from Observation.org and Artportalen. It evaluates four facets—classification performance, embedding quality, computational cost, and gradient activity—through per-dataset and per-species analyses, including knowledge-distillation ablations. The study finds Locality-based Vision Transformers (LBVT, notably ViTAE) delivering the best embedding quality and species-level separation, Vision Transformers (ViT, especially with distillation) offering strong inference efficiency, and CNNs providing robust baselines with a trade-off in embedding and long-range dependency modeling. The results inform practical guidance for biodiversity monitoring tasks, highlighting when to prioritize accuracy, embedding structure, or computational efficiency, and illustrating generalization challenges across data sources like Observation.org and Artportalen.

Abstract

With fine-grained classification, we identify unique characteristics to distinguish among classes of the same super-class. We are focusing on species recognition in Insecta, as they are critical for biodiversity monitoring and at the base of many ecosystems. With citizen science campaigns, billions of images are collected in the wild. Once these are labelled, experts can use them to create distribution maps. However, the labelling process is time-consuming, which is where computer vision comes in. The field of computer vision offers a wide range of algorithms, each with its strengths and weaknesses; how do we identify the algorithm that is in line with our application? To answer this question, we provide a full and detailed evaluation of nine algorithms among deep convolutional networks (CNN), vision transformers (ViT), and locality-based vision transformers (LBVT) on 4 different aspects: classification performance, embedding quality, computational cost, and gradient activity. We offer insights that we haven't yet had in this domain proving to which extent these algorithms solve the fine-grained tasks in Insecta. We found that the ViT performs the best on inference speed and computational cost while the LBVT outperforms the others on performance and embedding quality; the CNN provide a trade-off among the metrics.
Paper Structure (31 sections, 10 figures, 6 tables)

This paper contains 31 sections, 10 figures, 6 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Related work
  4. Methods
  5. Deep Learning Models
  6. Convolutional Neural Networks
  7. Vision Transformers
  8. Locality-based vision transformer
  9. Datasets
  10. Coleoptera_Obs
  11. Odonata_Obs
  12. Coleoptera_Art
  13. Odonata_Art
  14. Experimental configuration
  15. Preparation of the models
  16. ...and 16 more sections

Figures (10)

  • Figure 1: Images in each group belong to the same order, images in each row belong to the same genus, and each image represents a unique species from the Coleoptera and Odonata.
  • Figure 2: Metamorphism of Coleoptera (top row) and Odonata (bottom row) at different life stages.
  • Figure 3: Coleoptera_Obs
  • Figure 4: Odonata_Obs
  • Figure 6: The lines show the number of species (x-axis) by the accuracy top-1 (x-axis) obtained by each model. E.g.: in Fig. \ref{['fig:line_species_Coleo']} the EffNet classifies almost 2100 species with avgACC top-1 between $60\%$ and $70\%$.
  • ...and 5 more figures