Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention

Zhenyu Guo, Wenguang Chen

TL;DR

This work tackles the entanglement of knowledge and reasoning in Transformer models by proposing a modular architecture that decouples these components via a globally shared knowledge base $E$ and layer-specific transformations. It provides a rigorous theoretical result showing that the standard FFN is a closure of a generalized cross-attention to an implicit knowledge base, validating the proposed mechanism and offering a unified memory-based interpretation of FFNs. The model introduces a three-phase cross-attention for knowledge retrieval: sparse activation, knowledge-specific thresholding with $B1^l(E)$, and a transformation bias $b2^l$ to bridge semantic gaps. Together, these contributions lay a foundation for enhanced interpretability, adaptability, and scalable knowledge integration, including future work on external pluggable KBs and the associated computational trade-offs.

Abstract

Transformers have achieved remarkable success across diverse domains, but their monolithic architecture presents challenges in interpretability, adaptability, and scalability. This paper introduces a novel modular Transformer architecture that explicitly decouples knowledge and reasoning through a generalized cross-attention mechanism to a globally shared knowledge base with layer-specific transformations, specifically designed for effective knowledge retrieval. Critically, we provide a rigorous mathematical derivation demonstrating that the Feed-Forward Network (FFN) in a standard Transformer is a specialized case (a closure) of this generalized cross-attention, revealing its role in implicit knowledge retrieval and validating our design. This theoretical framework provides a new lens for understanding FFNs and lays the foundation for future research exploring enhanced interpretability, adaptability, and scalability, enabling richer interplay with external knowledge bases and other systems.

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention

TL;DR

This work tackles the entanglement of knowledge and reasoning in Transformer models by proposing a modular architecture that decouples these components via a globally shared knowledge base and layer-specific transformations. It provides a rigorous theoretical result showing that the standard FFN is a closure of a generalized cross-attention to an implicit knowledge base, validating the proposed mechanism and offering a unified memory-based interpretation of FFNs. The model introduces a three-phase cross-attention for knowledge retrieval: sparse activation, knowledge-specific thresholding with , and a transformation bias to bridge semantic gaps. Together, these contributions lay a foundation for enhanced interpretability, adaptability, and scalable knowledge integration, including future work on external pluggable KBs and the associated computational trade-offs.

Abstract

Transformers have achieved remarkable success across diverse domains, but their monolithic architecture presents challenges in interpretability, adaptability, and scalability. This paper introduces a novel modular Transformer architecture that explicitly decouples knowledge and reasoning through a generalized cross-attention mechanism to a globally shared knowledge base with layer-specific transformations, specifically designed for effective knowledge retrieval. Critically, we provide a rigorous mathematical derivation demonstrating that the Feed-Forward Network (FFN) in a standard Transformer is a specialized case (a closure) of this generalized cross-attention, revealing its role in implicit knowledge retrieval and validating our design. This theoretical framework provides a new lens for understanding FFNs and lays the foundation for future research exploring enhanced interpretability, adaptability, and scalability, enabling richer interplay with external knowledge bases and other systems.
Paper Structure (21 sections, 16 equations, 1 figure, 2 tables)

This paper contains 21 sections, 16 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Challenges of Monolithic Transformers
  3. The Decoder-Only Transformer Baseline
  4. Limitations of the Monolithic Architecture
  5. A Modular Architecture with Explicit Knowledge Decoupling
  6. Requirements for Explicit Knowledge Base
  7. The Proposed Architecture
  8. Generalized Cross-Attention for Knowledge Retrieval
  9. Phase 1: Selective Retrieval with Sparse Activation
  10. Phase 2: Knowledge-Specific Thresholding ("IF" Condition)
  11. Phase 3: Transformation Bias for Semantic Bridging
  12. FFN is a Closure of Generalized Cross-Attention
  13. Derivation of FFN from Generalized Cross-Attention
  14. Connection to Standard FFN
  15. Implications of the Equivalence
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Architectures for (a) standard decoder-only Transformer, and (b) our proposed modular Transformer (b) with generalized cross-attention and shared knowledge base.