From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring

TL;DR

This paper evaluates the performance of various XAI methods and examines how this approach facilitates severity quantification and growth monitoring and reveals that while the resulting segmentation masks may exhibit lower quality than those produced by supervised methods, they remain meaningful and enable severity monitoring, thus reducing substantial labeling costs.

Abstract

Monitoring surface cracks in infrastructure is crucial for structural health monitoring. Automatic visual inspection offers an effective solution, especially in hard-to-reach areas. Machine learning approaches have proven their effectiveness but typically require large annotated datasets for supervised training. Once a crack is detected, monitoring its severity often demands precise segmentation of the damage. However, pixel-level annotation of images for segmentation is labor-intensive. To mitigate this cost, one can leverage explainable artificial intelligence (XAI) to derive segmentations from the explanations of a classifier, requiring only weak image-level supervision. This paper proposes applying this methodology to segment and monitor surface cracks. We evaluate the performance of various XAI methods and examine how this approach facilitates severity quantification and growth monitoring. Results reveal that while the resulting segmentation masks may exhibit lower quality than those produced by supervised methods, they remain meaningful and enable severity monitoring, thus reducing substantial labeling costs.

Figures (10)

• Figure 1: Workflows for automated image-based crack segmentation and severity quantification. (a) Supervised semantic segmentation workflow. (b) Workflow of the proposed weakly-supervised methodology based on XAI and classifier explanations. This methodology allows to generate approximate segmentation masks and quantify severity while circumventing the high cost of pixel-level labeling of training images.
• Figure 2: Illustration of the post-processing steps using Integrated Gradients (top) and LRP (bottom) attribution maps. (1) Binarization (2) First morphological closing (3) Area opening (4) Second closing. The curve on the right shows the evolution of F1 score at each post-processing step.
• Figure 3: Overview of the NN-Explainer method stalder_what_2022 for damage classification with a negative (damage-free) class and $K$ positive (damage) classes. The explainer$\mathcal{E}$ learns to predict masks $\mathcal{S}$ for each class. Class $0$ represents damage-free samples. Masks of positive classes present in the input are merged using their element-wise maximum (denoted by $\vee$ in the diagram) to create the target mask $\mathbf{m}$ (and its inverse $\tilde{\mathbf{m}}$), while masks of other positive classes that are not present form the non-target mask $\mathbf{n}$. In our application, there is a single damage class for cracks, and the non-target mask does not play a role. The explanandum$\mathcal{F}$ (i.e., the model to be explained) is frozen.
• Figure 4: (left) Example negative (damage-free) image. (middle) Example positive (cracked) image. (right) Overlay of the ground-truth crack segmentation mask in red.
• Figure 5: Comparison between DeepLift attributions obtained with (b) the standard zero baseline and (d) our proposed baseline based on a sample of damage-free images. Attributions are more focused on the crack with our baseline. The behavior is similar for other attribution methods using a baseline image or distribution (e.g., Integrated Gradients).