Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LGM: Enhancing Large Language Models with Conceptual Meta-Relations and Iterative Retrieval

Wenchang Lei, Ping Zou, Yue Wang, Feng Sun, Lei Zhao

TL;DR

The paper tackles the problem of LLMs misinterpreting ambiguous or misaligned terms by introducing the Language Graph Model (LGM), which uses meta-relations—inheritance, alias, and composition—to define concepts more clearly. It combines a dual-graph representation (SRG for syntactic structure and CRG for concept relations), a two-phase Learning and Concept Iterative Retrieval workflow, and a reflection mechanism to validate extracted relations, enabling long-text and multi-hop reasoning without context-window limits. Through Concept Expansion and parallel retrieval with iterative merging, LGM reduces noise from raw passages and bridges dependencies across hops. Empirical results on HotpotQA and Musique show consistent improvements over state-of-the-art RAG baselines, demonstrating strong benefits for concept-centric retrieval in complex QA tasks. The work also provides a suite of prompts and datasets to support meta-relation extraction, reflection validation, and LGM reasoning, contributing to scalable, explainable retrieval-augmented reasoning for large language models.

Abstract

Large language models (LLMs) exhibit strong semantic understanding, yet struggle when user instructions involve ambiguous or conceptually misaligned terms. We propose the Language Graph Model (LGM) to enhance conceptual clarity by extracting meta-relations-inheritance, alias, and composition-from natural language. The model further employs a reflection mechanism to validate these meta-relations. Leveraging a Concept Iterative Retrieval Algorithm, these relations and related descriptions are dynamically supplied to the LLM, improving its ability to interpret concepts and generate accurate responses. Unlike conventional Retrieval-Augmented Generation (RAG) approaches that rely on extended context windows, our method enables large language models to process texts of any length without the need for truncation. Experiments on standard benchmarks demonstrate that the LGM consistently outperforms existing RAG baselines.

LGM: Enhancing Large Language Models with Conceptual Meta-Relations and Iterative Retrieval

TL;DR

The paper tackles the problem of LLMs misinterpreting ambiguous or misaligned terms by introducing the Language Graph Model (LGM), which uses meta-relations—inheritance, alias, and composition—to define concepts more clearly. It combines a dual-graph representation (SRG for syntactic structure and CRG for concept relations), a two-phase Learning and Concept Iterative Retrieval workflow, and a reflection mechanism to validate extracted relations, enabling long-text and multi-hop reasoning without context-window limits. Through Concept Expansion and parallel retrieval with iterative merging, LGM reduces noise from raw passages and bridges dependencies across hops. Empirical results on HotpotQA and Musique show consistent improvements over state-of-the-art RAG baselines, demonstrating strong benefits for concept-centric retrieval in complex QA tasks. The work also provides a suite of prompts and datasets to support meta-relation extraction, reflection validation, and LGM reasoning, contributing to scalable, explainable retrieval-augmented reasoning for large language models.

Abstract

Large language models (LLMs) exhibit strong semantic understanding, yet struggle when user instructions involve ambiguous or conceptually misaligned terms. We propose the Language Graph Model (LGM) to enhance conceptual clarity by extracting meta-relations-inheritance, alias, and composition-from natural language. The model further employs a reflection mechanism to validate these meta-relations. Leveraging a Concept Iterative Retrieval Algorithm, these relations and related descriptions are dynamically supplied to the LLM, improving its ability to interpret concepts and generate accurate responses. Unlike conventional Retrieval-Augmented Generation (RAG) approaches that rely on extended context windows, our method enables large language models to process texts of any length without the need for truncation. Experiments on standard benchmarks demonstrate that the LGM consistently outperforms existing RAG baselines.

Paper Structure

This paper contains 33 sections, 5 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Knowledge Scope and Expandable Knowledge
  4. Vector-Database-Based RAG
  5. Knowledge-Graph-Based RAG
  6. Context-Sentence Injection
  7. Concept-Sentence Injection
  8. Language Graph Model
  9. Workflow
  10. Learning
  11. NLP Processing
  12. Relation Extraction
  13. Reflection
  14. Concept Iterative Retrieval
  15. Concept Expansion
  16. ...and 18 more sections

Figures (5)

  • Figure 1: Cognitive capacity and expandable knowledge
  • Figure 2: The left of figure is structure of the syntactic relation graph (SRG) and the right is structure of the concept relation graph (CRG)
  • Figure 3: Workflow of the Language Graph Model
  • Figure 4: Concept Iterative Retrieval
  • Figure 5: HotpotQA on DeepSeek v3-0324 with varying maximum input size