Visual Localization via Semantic Structures in Autonomous Photovoltaic Power Plant Inspection

Abstract

Inspection systems utilizing unmanned aerial vehicles (UAVs) equipped with thermal cameras are increasingly popular for the maintenance of photovoltaic (PV) power plants. However, automation of the inspection task is a challenging problem as it requires precise navigation to capture images from optimal distances and viewing angles. This paper presents a novel localization pipeline that directly integrates PV module detection with UAV navigation, allowing precise positioning during inspection. Detections are used to identify the power plant structures in the image and associate these with the power plant model. We define visually recognizable anchor points for the initial association and use object tracking to discern global associations. We present three distinct methods for visual segmentation of PV modules based on traditional computer vision, deep learning, and their fusion, and we evaluate their performance in relation to the proposed localization pipeline. The presented methods were verified and evaluated using custom aerial inspection data sets, demonstrating their robustness and applicability for real-time navigation. Additionally, we evaluate the influence of the power plant model's precision on the localization methods.

Figures (22)

• Figure 1: (a) The DJI Matrice 300 drone during PV power plant inspection. A single PV module is highlighted in purple. (b) An example trajectory from PV inspection.
• Figure 2: Influence of Sun reflection in varying daytime.
• Figure 3: Navigation and inspection pipeline. Three different methods can be used for module detection.
• Figure 4: YOLO PV module detection and segmentation. (a) Original image. (b) Mask instance segmentation. (c) Bounding box detection.
• Figure 5: Individual phases of U-Net module detection. (a) U-Net based mask segmentation. (b) Detected modules.