Tracking and triangulating firefly flashes in field recordings

TL;DR

A training dataset and trained neural networks for reliable flash classification are provided and this robust tracking enables a new calibration-free method for the 3D reconstruction of flash occurrences from stereoscopic 360-degree videos.

Abstract

Identifying firefly flashes from other bright features in nature images is complicated. I provide a training dataset and trained neural networks for reliable flash classification. The training set consists of thousands of cropped images (patches) extracted by manual labeling from video recordings of fireflies in their natural habitat. The trained network appears as considerably more reliable to differentiate flashes from other sources of light compared to traditional methods relying solely on intensity thresholding. This robust tracking enables a new calibration-free method for the 3D reconstruction of flash occurrences from stereoscopic 360-degree videos, which I also present here.

Figures (4)

• Figure 1: In suboptimal recording conditions, there is a lot of bright artifacts in images. Intensity thresholding methods usually fail to isolate only flashes (inside red circles) from other bright spots, especially between tree leaves. A trained |net|, however, succeeds.
• Figure 2: Example of $65 \times 65$ pixels$^2$ RGB patches. Top row: flashes. Bottom row: bright artifacts.
• Figure 3: Comparison of flash tracking with and without net in a noisy recording. Without net classification, even with ABS, the tracker picks up plenty of false positives (black line), here especially because of wind moving vegetation. In contrast, by applying net, only true flashes are conserved (orange line).
• Figure 4: 3D reconstruction of a firefly swarm (10min recording).