Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLMN: Concept based Language Models via Neural Symbolic Reasoning

Yibo Yang

TL;DR

The paper tackles the interpretability gap in NLP by introducing CLMN, a neural-symbolic framework that leverages continuous concept embeddings and fuzzy-logic reasoning to model dynamic concept interactions without sacrificing performance. By integrating a Concept Layer and two neural-symbolic modules (Concept Polarity and Concept Relevance Networks), CLMN generates semantically meaningful explanations while maintaining competitive accuracy across multiple PLMs. Experiments on an augmented restaurant-review sentiment dataset demonstrate strong concept alignment and actionable explanations, with ablations highlighting the complementary roles of the concept and neural-symbolic components. The work suggests a practical path toward transparent, trustworthy NLP systems in domains like healthcare and finance, with opportunities for hierarchical reasoning and multilingual applicability.

Abstract

Deep learning has advanced NLP, but interpretability remains limited, especially in healthcare and finance. Concept bottleneck models tie predictions to human concepts in vision, but NLP versions either use binary activations that harm text representations or latent concepts that weaken semantics, and they rarely model dynamic concept interactions such as negation and context. We introduce the Concept Language Model Network (CLMN), a neural-symbolic framework that keeps both performance and interpretability. CLMN represents concepts as continuous, human-readable embeddings and applies fuzzy-logic reasoning to learn adaptive interaction rules that state how concepts affect each other and the final decision. The model augments original text features with concept-aware representations and automatically induces interpretable logic rules. Across multiple datasets and pre-trained language models, CLMN achieves higher accuracy than existing concept-based methods while improving explanation quality. These results show that integrating neural representations with symbolic reasoning in a unified concept space can yield practical, transparent NLP systems.

CLMN: Concept based Language Models via Neural Symbolic Reasoning

TL;DR

The paper tackles the interpretability gap in NLP by introducing CLMN, a neural-symbolic framework that leverages continuous concept embeddings and fuzzy-logic reasoning to model dynamic concept interactions without sacrificing performance. By integrating a Concept Layer and two neural-symbolic modules (Concept Polarity and Concept Relevance Networks), CLMN generates semantically meaningful explanations while maintaining competitive accuracy across multiple PLMs. Experiments on an augmented restaurant-review sentiment dataset demonstrate strong concept alignment and actionable explanations, with ablations highlighting the complementary roles of the concept and neural-symbolic components. The work suggests a practical path toward transparent, trustworthy NLP systems in domains like healthcare and finance, with opportunities for hierarchical reasoning and multilingual applicability.

Abstract

Deep learning has advanced NLP, but interpretability remains limited, especially in healthcare and finance. Concept bottleneck models tie predictions to human concepts in vision, but NLP versions either use binary activations that harm text representations or latent concepts that weaken semantics, and they rarely model dynamic concept interactions such as negation and context. We introduce the Concept Language Model Network (CLMN), a neural-symbolic framework that keeps both performance and interpretability. CLMN represents concepts as continuous, human-readable embeddings and applies fuzzy-logic reasoning to learn adaptive interaction rules that state how concepts affect each other and the final decision. The model augments original text features with concept-aware representations and automatically induces interpretable logic rules. Across multiple datasets and pre-trained language models, CLMN achieves higher accuracy than existing concept-based methods while improving explanation quality. These results show that integrating neural representations with symbolic reasoning in a unified concept space can yield practical, transparent NLP systems.

Paper Structure

This paper contains 27 sections, 16 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pretrained Language Models
  4. Concept Bottleneck Model
  5. Neural Symbolic Reasoning
  6. Methods
  7. Preliminaries
  8. Pretrained Language Model
  9. Concept Layer and Concept Embedding
  10. Concept Layer
  11. Concept Representation.
  12. Concept Activation and Embedding Computation.
  13. Integration with the Prediction Model.
  14. Neural-Symbolic Concept Reasoning
  15. Concept Polarity Network.
  16. ...and 12 more sections

Figures (2)

  • Figure 1: This flowchart details a text - classification framework. An input sentence is input into a PLM. The Concept Layer predicts aspects like food. Its results, combined with PLM's, yield a sentiment score. Concepts also flow to the Concept Reasoning Layer, constraining internal model explanations.
  • Figure 2: Interpretability Example