Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating the method reproducibility of deep learning models in the biodiversity domain

Waqas Ahmed, Vamsi Krishna Kommineni, Birgitta König-Ries, Jitendra Gaikwad, Luiz Gadelha, Sheeba Samuel

TL;DR

This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain with a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages.

Abstract

Artificial Intelligence (AI) is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility in AI-driven biodiversity research is crucial for fostering transparency, verifying results, and promoting the credibility of ecological findings.This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain. We design a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages. We define ten variables essential for method reproducibility, divided into four categories: resource requirements, methodological information, uncontrolled randomness, and statistical considerations. These categories subsequently serve as the basis for defining different levels of reproducibility. We manually extract the availability of these variables from a curated dataset comprising 61 publications identified using the keywords provided by biodiversity experts. Our study shows that the dataset is shared in 47% of the publications; however, a significant number of the publications lack comprehensive information on deep learning methods, including details regarding randomness.

Evaluating the method reproducibility of deep learning models in the biodiversity domain

TL;DR

This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain with a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages.

Abstract

Artificial Intelligence (AI) is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility in AI-driven biodiversity research is crucial for fostering transparency, verifying results, and promoting the credibility of ecological findings.This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain. We design a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages. We define ten variables essential for method reproducibility, divided into four categories: resource requirements, methodological information, uncontrolled randomness, and statistical considerations. These categories subsequently serve as the basis for defining different levels of reproducibility. We manually extract the availability of these variables from a curated dataset comprising 61 publications identified using the keywords provided by biodiversity experts. Our study shows that the dataset is shared in 47% of the publications; however, a significant number of the publications lack comprehensive information on deep learning methods, including details regarding randomness.
Paper Structure (15 sections, 7 figures)

This paper contains 15 sections, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Results
  5. Discussion
  6. Conclusion
  7. Data availability statement

Figures (7)

  • Figure 1: Number of publications considered for collecting the binary responses as per the reproducibility criteria based on year
  • Figure 2: Publisher information for the 61 publications selected for collecting the binary responses as per the reproducibility criteria
  • Figure 3: Number of levels along with the categories covered for respective levels
  • Figure 4: Binary responses of the considered reproducible variables in four categories 1) Resources 2) Methodological information 3) Randomness 4) Statistical consideration for selected research publication
  • Figure 5: Distribution of selected research publications along four categories
  • ...and 2 more figures