Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Language Concepts for Interpreting Foundation Language Models

Hengyi Wang, Shiwei Tan, Zhiqing Hong, Desheng Zhang, Hao Wang

TL;DR

A formal definition of conceptual interpretation is provided and a variational Bayesian framework is proposed, dubbed VAriational Language Concept (VALC), to go beyond word-level interpretations and provide concept-level interpretations of FLM predictions.

Abstract

Foundation Language Models (FLMs) such as BERT and its variants have achieved remarkable success in natural language processing. To date, the interpretability of FLMs has primarily relied on the attention weights in their self-attention layers. However, these attention weights only provide word-level interpretations, failing to capture higher-level structures, and are therefore lacking in readability and intuitiveness. To address this challenge, we first provide a formal definition of conceptual interpretation and then propose a variational Bayesian framework, dubbed VAriational Language Concept (VALC), to go beyond word-level interpretations and provide concept-level interpretations. Our theoretical analysis shows that our VALC finds the optimal language concepts to interpret FLM predictions. Empirical results on several real-world datasets show that our method can successfully provide conceptual interpretation for FLMs.

Variational Language Concepts for Interpreting Foundation Language Models

TL;DR

A formal definition of conceptual interpretation is provided and a variational Bayesian framework is proposed, dubbed VAriational Language Concept (VALC), to go beyond word-level interpretations and provide concept-level interpretations of FLM predictions.

Abstract

Foundation Language Models (FLMs) such as BERT and its variants have achieved remarkable success in natural language processing. To date, the interpretability of FLMs has primarily relied on the attention weights in their self-attention layers. However, these attention weights only provide word-level interpretations, failing to capture higher-level structures, and are therefore lacking in readability and intuitiveness. To address this challenge, we first provide a formal definition of conceptual interpretation and then propose a variational Bayesian framework, dubbed VAriational Language Concept (VALC), to go beyond word-level interpretations and provide concept-level interpretations. Our theoretical analysis shows that our VALC finds the optimal language concepts to interpret FLM predictions. Empirical results on several real-world datasets show that our method can successfully provide conceptual interpretation for FLMs.
Paper Structure (1 section, 1 figure)

This paper contains 1 section, 1 figure.

Table of Contents

  1. Introduction

Figures (1)

  • Figure 1: Visualization of VALC's learned concepts. A document consists of two sentences. The task is to decide whether 'Sentence 1' paraphrases 'Sentence 2'. Left: Dataset-level concepts for MRPC dataset with $3$ concepts and their nearest word embeddings. Middle: Document-level concept strength, showing that this document is mostly related to Concept 20 and Concept 24. Right: Word-level concepts, where the FLM correctly predicts the label to be 'True', and VALC interprets that this is because the both sentences consist of words with Concept 24, i.e., Politics.