Targeted Visualization of the Backbone of Encoder LLMs

TL;DR

DeepView is demonstrated how to apply it to BERT-based NLP classifiers and investigated its usability in this domain, including settings with adversarially perturbed input samples and pre-trained, fine-tuned, and multi-task models.

Abstract

Attention based Large Language Models (LLMs) are the state-of-the-art in natural language processing (NLP). The two most common architectures are encoders such as BERT, and decoders like the GPT models. Despite the success of encoder models, on which we focus in this work, they also bear several risks, including issues with bias or their susceptibility for adversarial attacks, signifying the necessity for explainable AI to detect such issues. While there does exist various local explainability methods focusing on the prediction of single inputs, global methods based on dimensionality reduction for classification inspection, which have emerged in other domains and that go further than just using t-SNE in the embedding space, are not widely spread in NLP. To reduce this gap, we investigate the application of DeepView, a method for visualizing a part of the decision function together with a data set in two dimensions, to the NLP domain. While in previous work, DeepView has been used to inspect deep image classification models, we demonstrate how to apply it to BERT-based NLP classifiers and investigate its usability in this domain, including settings with adversarially perturbed input samples and pre-trained, fine-tuned, and multi-task models.

Figures (5)

• Figure 1: DeepView of the Pre-Trained BERT with classification head trained on SST2, a binary classification dataset; the background colors correspond to either class label; the color intensity is a proxy for classification certainty; one dot corresponds to a single embedded sentence. $\lambda$ encodes different amounts of discriminative information.
• Figure 2: DeepView of the Multi-Task BERT and SST2 Fine-tuned BERT Model's embedding space with respect to the SST2 dataset. Figure \ref{['subfig:multitask_bert']} also contains an adversarially attacked data point. The bottom left of the cyan region displays an area of high uncertainty which we investigate in Section \ref{['sec:exper_adv']}.
• Figure 3: Left: This image is zoomed into the uncertain region in the bottom left of the cyan region of Figure \ref{['subfig:multitask_bert']}. The numbers correspond to the selection order of the points and to the sentences on the right. We selected points from left to right because the background indicates a high level of uncertainty in the left. The fifth point selected reveals the adversarial attack.
• Figure 4: Left: DeepView of a kNN classifier on the embedding space of the multi-task model which is trained to differentiate between the datasets. Each point and region are colored according to respective dataset. Right: Confusion matrix of the same kNN classifier.
• Figure 5: $Q_{NN}(k)$ and LCMC curves of models trained on SST2 in high (\ref{['subfig:high_dim']}) and low (\ref{['subfig:low_dim']}) dimensions. Visually, we can see each of the curves are very similar between high and low dimensions. The similarity is further proven by the AUC scores located in the bottom right of each figure.