SubstationAI: Multimodal Large Model-Based Approaches for Analyzing Substation Equipment Faults
Jinzhi Wang, Qinfeng Song, Lidong Qian, Haozhou Li, Qinke Peng, Jiangbo Zhang
TL;DR
SubstationAI tackles substation fault analysis by fusing image features with textual prompts to generate structured fault reports, formalized as $R = \text{GenReport}(\text{Fuse}(\text{ImgFeat}(I), \text{TxtEnc}(P)))$, yielding $T$ (fault type), $C$ (fault causes), and $S$ (repair guidance). It builds a 40,000-sample dataset via public defect data, GPT-4 report generation, expert curation, and image-to-video augmentation (EasyAnimate) to diversify inputs. The approach uses LoRA fine-tuning on LLAVA1.5-7B and a knowledge enhancement pipeline that integrates a domain-specific fault diagnosis knowledge base with BERT-driven keyword extraction. Empirical results show SubstationAI outperforms baselines including GPT-4 across accuracy, clarity, completeness, and practicality, offering a practical, scalable tool for fault diagnosis, repair planning, and preventive maintenance in power systems.
Abstract
The reliability of substation equipment is crucial to the stability of power systems, but traditional fault analysis methods heavily rely on manual expertise, limiting their effectiveness in handling complex and large-scale data. This paper proposes a substation equipment fault analysis method based on a multimodal large language model (MLLM). We developed a database containing 40,000 entries, including images, defect labels, and analysis reports, and used an image-to-video generation model for data augmentation. Detailed fault analysis reports were generated using GPT-4. Based on this database, we developed SubstationAI, the first model dedicated to substation fault analysis, and designed a fault diagnosis knowledge base along with knowledge enhancement methods. Experimental results show that SubstationAI significantly outperforms existing models, such as GPT-4, across various evaluation metrics, demonstrating higher accuracy and practicality in fault cause analysis, repair suggestions, and preventive measures, providing a more advanced solution for substation equipment fault analysis.