Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D-Plotting Algorithm for Insects using YOLOv5

Daisuke Mori, Hiroki Hayami, Yasufumi Fujimoto, Isao Goto

TL;DR

This work addresses the need for automated, three-dimensional tracking of insects in experimental environments using inexpensive hardware. It integrates a YOLOv5-based 2D detector with a multi-stage 3D reconstruction pipeline that employs homography, scaling, and world-coordinate remapping, augmented by a depth-error correction mechanism. The approach demonstrates quantitative 3D plotting accuracy and practical visualization, showing that a substantial fraction of detected insects can be represented in 3D with a measured error of about $2.59$ cm. The method lowers barriers to detailed ecological analyses by combining accessible hardware with principled geometric corrections, though it notes limitations related to detector accuracy, insect density, and camera stability.

Abstract

In ecological research, accurately collecting spatiotemporal position data is a fundamental task for understanding the behavior and ecology of insects and other organisms. In recent years, advancements in computer vision techniques have reached a stage of maturity where they can support, and in some cases, replace manual observation. In this study, a simple and inexpensive method for monitoring insects in three dimensions (3D) was developed so that their behavior could be observed automatically in experimental environments. The main achievements of this study have been to create a 3D monitoring algorithm using inexpensive cameras and other equipment to design an adjusting algorithm for depth error, and to validate how our plotting algorithm is quantitatively precise, all of which had not been realized in conventional studies. By offering detailed 3D visualizations of insects, the plotting algorithm aids researchers in more effectively comprehending how insects interact within their environments.

3D-Plotting Algorithm for Insects using YOLOv5

TL;DR

This work addresses the need for automated, three-dimensional tracking of insects in experimental environments using inexpensive hardware. It integrates a YOLOv5-based 2D detector with a multi-stage 3D reconstruction pipeline that employs homography, scaling, and world-coordinate remapping, augmented by a depth-error correction mechanism. The approach demonstrates quantitative 3D plotting accuracy and practical visualization, showing that a substantial fraction of detected insects can be represented in 3D with a measured error of about cm. The method lowers barriers to detailed ecological analyses by combining accessible hardware with principled geometric corrections, though it notes limitations related to detector accuracy, insect density, and camera stability.

Abstract

In ecological research, accurately collecting spatiotemporal position data is a fundamental task for understanding the behavior and ecology of insects and other organisms. In recent years, advancements in computer vision techniques have reached a stage of maturity where they can support, and in some cases, replace manual observation. In this study, a simple and inexpensive method for monitoring insects in three dimensions (3D) was developed so that their behavior could be observed automatically in experimental environments. The main achievements of this study have been to create a 3D monitoring algorithm using inexpensive cameras and other equipment to design an adjusting algorithm for depth error, and to validate how our plotting algorithm is quantitatively precise, all of which had not been realized in conventional studies. By offering detailed 3D visualizations of insects, the plotting algorithm aids researchers in more effectively comprehending how insects interact within their environments.
Paper Structure (24 sections, 6 equations, 10 figures, 1 table)

This paper contains 24 sections, 6 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Hardware Details
  4. Insect Detection using Deep Learning
  5. Dataset Collection and Labeling
  6. Training
  7. Insect Location Identification in Images
  8. 3D Insect Spatial Reconstruction using 2D coordinates
  9. Minimizing Positioning Error
  10. Homography
  11. Scaling Transformation
  12. Re-mapping to the World Coordinate System
  13. Depth Error Modification
  14. Result
  15. Insect Detection via YOLOv5 Model
  16. ...and 9 more sections

Figures (10)

  • Figure 1: Set up of the observation system (a) and a frame from the recorded videos (b). White stickers attached to the black framework (Grid A) are marks for algorithm calculation.
  • Figure 2: Chrysolina virgata.
  • Figure 3: The concept of our algorithm and the dataflow.
  • Figure 4: Process of minimizing positioning error. (a) The images of Grid A are not same due to the difference of camera’s angle and position. (b) The images are adjusted for the shape and size of each sub-area. (c) The sub-areas are combined to be one rectangle. These rectangles of each of the images have same size and shape.
  • Figure 5: Image of one side of Grid A and the sub-areas that undergo homography and scaling transformation to minimize positioning error. The origin for each respective sub-area is numbered accordingly.
  • ...and 5 more figures