Distant Object Localisation from Noisy Image Segmentation Sequences

TL;DR

The results suggest that combining the proposed methods with pre-existing image segmentation models and drone-carried computational resources yields a reliable system for drone-based wildfire monitoring.

Abstract

3D object localisation based on a sequence of camera measurements is essential for safety-critical surveillance tasks, such as drone-based wildfire monitoring. Localisation of objects detected with a camera can typically be solved with specialised sensor configurations or 3D scene reconstruction. However, in the context of distant objects or tasks limited by the amount of available computational resources, neither solution is feasible. In this paper, we show that the task can be solved with either multi-view triangulation or particle filters, with the latter also providing shape and uncertainty estimates. We studied the solutions using 3D simulation and drone-based image segmentation sequences with global navigation satellite system (GNSS) based camera pose estimates. The results suggest that combining the proposed methods with pre-existing image segmentation models and drone-carried computational resources yields a reliable system for drone-based wildfire monitoring. The proposed solutions are independent of the detection method, also enabling quick adaptation to similar tasks.

Figures (6)

• Figure 1: We propose a hybrid approach for localising distant objects/events (such as wildfire smoke) from sequences of frames and GNSS-estimated poses from a moving RGB camera (example frames with masked smoke shown on top-right).
• Figure 2: Noisy single target simulation results. From top to bottom: Simulated camera translation from the start of the sequence, noiseless single target simulation sample frames, fully noisy simulation samples, RMSEs of the single target simulation experiments over the camera translation with a logarithmic RMSE axis. The plot highlights the smooth convergence of the particle filter predictions compared to those of the multi-view triangulation.
• Figure 3: Convergence of the particle ratio over the camera translation in simulation in different noise scenarios. The types of noise are additive as in Table \ref{['tab:simple_results']}. The almost equal ratio curves show that the particle filter converges towards the right region despite the various noise.
• Figure 4: The first empirical test sequence of the telecommunication mast target. On the top, the drone-captured RGB images and below, the used segments from edge detection. For visualisation, the segment has been dilated for an additional ten steps, and the images have been centre-cropped to half width and height.
• Figure 5: The second empirical test sequence of the industrial smoke cloud target. On top, the drone-captured RGB images and below, the used segments from SAM 3. For visualisation, the images have been centre-cropped to half width and height.