Prototype-based Explainable Neural Networks with Channel-specific Reasoning for Geospatial Learning Tasks

TL;DR

This work introduces an intrinsically interpretable, channel-specific prototype-based neural network tailored for multi-channel geospatial raster data. By employing a shared encoder per channel, channel-wise prototype layers, and a final classifier, the model yields both local, instance-level explanations via prototype similarities and global explanations via channel importance and prototype frequency. The approach is validated on a synthetic MNIST-derived task, Madden-Julian Oscillation climate classification, and Euro-SAT land-use classification, achieving competitive accuracy while providing meaningful, physically interpretable explanations tied to individual channels. The results demonstrate that channel-specific prototypes enhance transparency for geoscientific learning tasks and remain compatible with pre-trained encoders, offering a practical path toward trustworthy, interpretable deep learning in Earth and climate sciences.

Abstract

Explainable AI (XAI) is essential for understanding machine learning (ML) decision-making and ensuring model trustworthiness in scientific applications. Prototype-based XAI methods offer an intrinsically interpretable alternative to post-hoc approaches which often yield inconsistent explanations. Prototype-based XAI methods make predictions based on the similarity between inputs and learned prototypes that represent typical characteristics of target classes. However, existing prototype-based models are primarily designed for standard RGB image data and are not optimized for the distinct, variable-specific channels commonly found in geoscientific image and raster datasets. In this study, we develop a prototype-based XAI approach tailored for multi-channel geospatial data, where each channel represents a distinct physical environmental variable or spectral channel. Our approach enables the model to identify separate, channel-specific prototypical characteristics sourced from multiple distinct training examples that inform how these features individually and in combination influence model prediction while achieving comparable performance to standard neural networks. We demonstrate this method through two geoscientific case studies: (1) classification of Madden Julian Oscillation phases using multi-variable climate data and (2) land-use classification from multispectral satellite imagery. This approach produces both local (instance-level) and global (model-level) explanations for providing insights into feature-relevance across channels. By explicitly incorporating channel-prototypes into the prediction process, we discuss how this approach enhances the transparency and trustworthiness of ML models for geoscientific learning tasks.

Figures (10)

• Figure 1: Diagrammatic representation of (upper): a standard neural network with post hoc XAI applied to a trained model to produce an explanation and (lower): an intrinsically interpretable alternative model with prototype-based reasoning to produce an explanation
• Figure 2: (a) High-level architecture overview of the channel-specific prototype method. (b) Encoder: Input sample, $X^{(i)}$, is separated channel-wise and processed through the encoder, $E$, to produce an embedding for each individual channel (a single channel's embedding is displayed here for simplicity). (c) Prototype Learning Layer: Each channel-specific embedding, $Z_j^{(i)}$, is processed through its respective prototype layer, $P_j$, and compared against the set of $N$ prototypes for this channel to produce $N$ similarity grids, $S_j^{(i)}$. (a single channel's operations are displayed here for simplicity) (d) Output Layer: Maximum similarity values for each prototype are obtained and concatenated across all channels resulting in a $N \cdot C$ length vector and fed through the linear classifier to produce a prediction.
• Figure 3: (a) Example of synthetically generated sample. The left three images show three image channels separated. Image on right shows the sample with channels concatenated together (channels shown stacked for visualization purposes). Ground truth label: 2 (b) Test Sample 98's prototype score distribution (on logarithmic scale, capped below at $10^{-6}$) for prediction of class "2". Prototype scores are grouped by channel where blue, orange and green bars represent Channel 1, 2, and 3 prototypes respectively and sorted in ascending order. The dashed outline box highlights the top three scoring prototypes that contributed to this prediction visualized in Panel c. (c) Top three scoring prototypes shown within the red outline boxes capturing "2" digit pattern displayed along with the original training image from which the prototype originates. Below each prototype is its associated location scaling indicating "side" where similarity to the prototype is weighted strongly (red indicates negative relevance, grey indicates positive relevance). Explanation of model prediction: The model predicts that the image belongs to class "2" because of a high similarity to the "2" digit pattern in the right side of channel 2 in training image 8613, which makes the largest contribution to the final prediction.
• Figure 4: MNIST Case Study final linear layer weight matrix (rows and columns permuted for visual clarity) separated into even digit class labels (0,2,4,6,8; left panel) and odd digit class labels (1,3,5,7,9; right panel). The matrix shows the weights associated with the 50 prototypes from each channel (top: channel 1, middle: channel 2, bottom: channel 3). Strong positive (red) and negative (blue) weights are predominantly assigned to channel 2 for even class labels and to channel 3 for odd class labels while near zero (white) weights appear for prototypes from channels that are irrelevant to the corresponding class.
• Figure 5: In the top row, across the test set, the most frequent high scoring model-identified prototypes (digit or "blank" pattern) are shown for each class (0-9). Information on the specific channel and training sample from which the prototype originates from is included. Bottom row shows each prototype's associated location scaling grid which shows the 'side' (left or right) of an input sample where the prototype pattern is more relevant where gray shading indicates positive relevance and red shading indicates negative relevance. For even-numbered classes, the prototype digit pattern is more relevant on the right hand side and for vice versa for odd-numbered classes, with the exception of class label 5 due to the "blank" prototype identified.