Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AutoMathKG: The automated mathematical knowledge graph based on LLM and vector database

Rong Bian, Yu Geng, Zijian Yang, Bing Cheng

TL;DR

AutoMathKG addresses the challenge of building a comprehensive, updatable mathematical knowledge graph from natural language sources. It integrates diverse sources (ProofWiki, textbooks, arXiv, TheoremQA) and augments entities with LLMs, while MathVD enables fast similarity search via two embedding strategies. The system combines automatic knowledge completion by a Math LLM and automatic knowledge fusion using MathVD and LLM, demonstrating strong reachability and mathematical reasoning capabilities. This work provides a scalable, automated pipeline for expanding structured mathematical knowledge with potential benefits for AI-powered mathematics research and applications.

Abstract

A mathematical knowledge graph (KG) presents knowledge within the field of mathematics in a structured manner. Constructing a math KG using natural language is an essential but challenging task. There are two major limitations of existing works: first, they are constrained by corpus completeness, often discarding or manually supplementing incomplete knowledge; second, they typically fail to fully automate the integration of diverse knowledge sources. This paper proposes AutoMathKG, a high-quality, wide-coverage, and multi-dimensional math KG capable of automatic updates. AutoMathKG regards mathematics as a vast directed graph composed of Definition, Theorem, and Problem entities, with their reference relationships as edges. It integrates knowledge from ProofWiki, textbooks, arXiv papers, and TheoremQA, enhancing entities and relationships with large language models (LLMs) via in-context learning for data augmentation. To search for similar entities, MathVD, a vector database, is built through two designed embedding strategies using SBERT. To automatically update, two mechanisms are proposed. For knowledge completion mechanism, Math LLM is developed to interact with AutoMathKG, providing missing proofs or solutions. For knowledge fusion mechanism, MathVD is used to retrieve similar entities, and LLM is used to determine whether to merge with a candidate or add as a new entity. A wide range of experiments demonstrate the advanced performance and broad applicability of the AutoMathKG system, including superior reachability query results in MathVD compared to five baselines and robust mathematical reasoning capability in Math LLM.

AutoMathKG: The automated mathematical knowledge graph based on LLM and vector database

TL;DR

AutoMathKG addresses the challenge of building a comprehensive, updatable mathematical knowledge graph from natural language sources. It integrates diverse sources (ProofWiki, textbooks, arXiv, TheoremQA) and augments entities with LLMs, while MathVD enables fast similarity search via two embedding strategies. The system combines automatic knowledge completion by a Math LLM and automatic knowledge fusion using MathVD and LLM, demonstrating strong reachability and mathematical reasoning capabilities. This work provides a scalable, automated pipeline for expanding structured mathematical knowledge with potential benefits for AI-powered mathematics research and applications.

Abstract

A mathematical knowledge graph (KG) presents knowledge within the field of mathematics in a structured manner. Constructing a math KG using natural language is an essential but challenging task. There are two major limitations of existing works: first, they are constrained by corpus completeness, often discarding or manually supplementing incomplete knowledge; second, they typically fail to fully automate the integration of diverse knowledge sources. This paper proposes AutoMathKG, a high-quality, wide-coverage, and multi-dimensional math KG capable of automatic updates. AutoMathKG regards mathematics as a vast directed graph composed of Definition, Theorem, and Problem entities, with their reference relationships as edges. It integrates knowledge from ProofWiki, textbooks, arXiv papers, and TheoremQA, enhancing entities and relationships with large language models (LLMs) via in-context learning for data augmentation. To search for similar entities, MathVD, a vector database, is built through two designed embedding strategies using SBERT. To automatically update, two mechanisms are proposed. For knowledge completion mechanism, Math LLM is developed to interact with AutoMathKG, providing missing proofs or solutions. For knowledge fusion mechanism, MathVD is used to retrieve similar entities, and LLM is used to determine whether to merge with a candidate or add as a new entity. A wide range of experiments demonstrate the advanced performance and broad applicability of the AutoMathKG system, including superior reachability query results in MathVD compared to five baselines and robust mathematical reasoning capability in Math LLM.
Paper Structure (56 sections, 7 equations, 12 figures, 21 tables)

This paper contains 56 sections, 7 equations, 12 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Mathematical data resource
  4. Knowledge graph and vector database
  5. In-context learning with LLMs
  6. AutoMathKG system overview
  7. Method of AutoMathKG construction
  8. Corpus collection
  9. ProofWiki
  10. TheoremQA
  11. Textbook
  12. ArXiv
  13. AutoMathKG structure
  14. Entity vertices
  15. Entity storage
  16. ...and 41 more sections

Figures (12)

  • Figure 1: Examples of entities and reference edges with tactics in AutoMathKG.
  • Figure 2: Model architecture of AutoMathKG construction for building Input KG from input text.
  • Figure 3: Model architecture of AutoMathKG automatic updates for knowledge completion by Math LLM and knowledge fusion by VD and LLM between Input KG and Existing KG.
  • Figure 4: The schema of entity storage in JSON format.
  • Figure 5: The diagram of all possible relationships between different types of entities.
  • ...and 7 more figures