Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigation

TL;DR

The paper tackles domain shift and data scarcity in photovoltaic defect detection from EL images. It introduces PDIG, a diffusion-based generator built on Stable Diffusion, featuring Semantic Concept Embedding, Lightweight Industrial Style Adaptor, and Text-Image Dual-Space Constraints to learn industrial defect concepts and inject domain-relevant styles. PDIG achieves superior realism and diversity, reducing the average FID by 19.16 points compared with the next-best method and enhancing downstream defect detection performance. Additionally, the Text-Image Dual-Space Constraints provide spatially aware generation and annotation priors, enabling semi-automatic labeling and improved localization of generated defects.

Abstract

Accurate defect detection of photovoltaic (PV) cells is critical for ensuring quality and efficiency in intelligent PV manufacturing systems. However, the scarcity of rich defect data poses substantial challenges for effective model training. While existing methods have explored generative models to augment datasets, they often suffer from instability, limited diversity, and domain shifts. To address these issues, we propose PDIG, a Photovoltaic Defect Image Generator based on Stable Diffusion (SD). PDIG leverages the strong priors learned from large-scale datasets to enhance generation quality under limited data. Specifically, we introduce a Semantic Concept Embedding (SCE) module that incorporates text-conditioned priors to capture the relational concepts between defect types and their appearances. To further enrich the domain distribution, we design a Lightweight Industrial Style Adaptor (LISA), which injects industrial defect characteristics into the SD model through cross-disentangled attention. At inference, we propose a Text-Image Dual-Space Constraints (TIDSC) module, enforcing the quality of generated images via positional consistency and spatial smoothing alignment. Extensive experiments demonstrate that PDIG achieves superior realism and diversity compared to state-of-the-art methods. Specifically, our approach improves Frechet Inception Distance (FID) by 19.16 points over the second-best method and significantly enhances the performance of downstream defect detection tasks.

Figures (9)

• Figure 1: The published ELES dataset exhibits issues of domain shift and instance shift. (a) Domain shift: variations exist in the resolution, background brightness, and grid line spacing of defect images across different production lines; Instance shift: the distribution of instances in the training dataset is inconsistent with that in the test data. For example, cracks in the training set appear as fine, linear defects, whereas those in the testing set are characterized by large-area damage with inconsistent shapes. (b) t-SNE dimensionality reduction and probability distribution visualization on the data from different production lines reveal data distribution deviations caused by endogenous shifts.
• Figure 2: The images generated by the SD V1.5 model with the prompt “PV defect images based on EL imaging”.
• Figure 3: The PDIG employs a two-stage generation approach. During the training stage, the Semantic Concept Embedding (SCE) module and the Industrial Style Adaptation (LISA) module learn image and text features, which are then fully integrated into the backbone network to enhance the image generation domain distribution. In the inference stage, the Text-Image Dual-Space Constraints (TIDSC) module is utilized to focus more attention on the specified bounding box regions, thereby enabling precise defect localization generation.
• Figure 4: Illustration of cross-disentangled attention between Text and Image.
• Figure 5: Distribution of defect categories in each production line of the dataset. It illustrates the significant differences in the proportion of data among the three groups and the distribution of defect quantities within each group. For example, the number of "scratch" defects in EL group3 is only 62, highlighting a noticeable numerical discrepancy.