Concept Layers: Enhancing Interpretability and Intervenability via LLM Conceptualization

TL;DR

This work introduces Concept Layers (CLs), an in-architecture approach to render LLMs interpretable and intervenable by projecting latent representations into an interpretable concept space $L_C$ via a non-trainable layer and reconstructing back into the model. A novel ontology-based, variance-guided concept-set generation selects a compact, task-appropriate $C$ without requiring labeled concept datasets. Empirical results on AG News, Yelp Polarity, and DBpedia demonstrate preserved or improved raw performance and high agreement with the original model, along with a functional intervenability interface to mitigate biases. The methodology preserves architectural compatibility, supports both task-specific and task-agnostic configurations, and offers practical avenues for debugging, bias mitigation, and safer deployment of NLP systems.

Abstract

The opaque nature of Large Language Models (LLMs) has led to significant research efforts aimed at enhancing their interpretability, primarily through post-hoc methods. More recent in-hoc approaches, such as Concept Bottleneck Models (CBMs), offer both interpretability and intervenability by incorporating explicit concept representations. However, these methods suffer from key limitations, including reliance on labeled concept datasets and significant architectural modifications that challenges re-integration into existing system pipelines. In this work, we introduce a new methodology for incorporating interpretability and intervenability into an existing model by integrating Concept Layers (CLs) into its architecture. Our approach projects the model's internal vector representations into a conceptual, explainable vector space before reconstructing and feeding them back into the model. Furthermore, we eliminate the need for a human-selected concept set by algorithmically searching an ontology for a set of concepts that can be either task-specific or task-agnostic. We evaluate CLs across multiple tasks, demonstrating that they maintain the original model's performance and agreement while enabling meaningful interventions. Additionally, we present a proof of concept showcasing an intervenability interface, allowing users to adjust model behavior dynamically, such as mitigating biases during inference.

Figures (1)

• Figure 1: Our methodology, visualized on the task of credit approval. The "Politics" concept can be attenuated in the conceptual space to mitigate bias before passing the vector back to the model.