Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transparent AI for Mathematics: Transformer-Based Large Language Models for Mathematical Entity Relationship Extraction with XAI

Tanjim Taharat Aurpa

TL;DR

This study formulates mathematical problem interpretation as a Mathematical Entity Relation Extraction task, where operands are treated as entities and operators as their relationships, and reveals how specific textual and mathematical features influence relation prediction.

Abstract

Mathematical text understanding is a challenging task due to the presence of specialized entities and complex relationships between them. This study formulates mathematical problem interpretation as a Mathematical Entity Relation Extraction (MERE) task, where operands are treated as entities and operators as their relationships. Transformer-based models are applied to automatically extract these relations from mathematical text, with Bidirectional Encoder Representations from Transformers (BERT) achieving the best performance, reaching an accuracy of 99.39%. To enhance transparency and trust in the model's predictions, Explainable Artificial Intelligence (XAI) is incorporated using Shapley Additive Explanations (SHAP). The explainability analysis reveals how specific textual and mathematical features influence relation prediction, providing insights into feature importance and model behavior. By combining transformer-based learning, a task-specific dataset, and explainable modeling, this work offers an effective and interpretable framework for MERE, supporting future applications in automated problem solving, knowledge graph construction, and intelligent educational systems.

Transparent AI for Mathematics: Transformer-Based Large Language Models for Mathematical Entity Relationship Extraction with XAI

TL;DR

This study formulates mathematical problem interpretation as a Mathematical Entity Relation Extraction task, where operands are treated as entities and operators as their relationships, and reveals how specific textual and mathematical features influence relation prediction.

Abstract

Mathematical text understanding is a challenging task due to the presence of specialized entities and complex relationships between them. This study formulates mathematical problem interpretation as a Mathematical Entity Relation Extraction (MERE) task, where operands are treated as entities and operators as their relationships. Transformer-based models are applied to automatically extract these relations from mathematical text, with Bidirectional Encoder Representations from Transformers (BERT) achieving the best performance, reaching an accuracy of 99.39%. To enhance transparency and trust in the model's predictions, Explainable Artificial Intelligence (XAI) is incorporated using Shapley Additive Explanations (SHAP). The explainability analysis reveals how specific textual and mathematical features influence relation prediction, providing insights into feature importance and model behavior. By combining transformer-based learning, a task-specific dataset, and explainable modeling, this work offers an effective and interpretable framework for MERE, supporting future applications in automated problem solving, knowledge graph construction, and intelligent educational systems.
Paper Structure (21 sections, 7 equations, 12 figures, 3 tables)

This paper contains 21 sections, 7 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Methodology
  4. Data Collection and Preprocessing
  5. Model Building, Training and Evaluation
  6. Proposed BERT Model
  7. Hyperparameter Tuning
  8. Model Evaluation
  9. Model Explainability
  10. Proposed Approach
  11. Code Availability
  12. Results
  13. Discussion
  14. Limitation
  15. Conclusion and Future Work
  16. ...and 6 more sections

Figures (12)

  • Figure 1: Step-by-step pipeline for constructing the Mathematical Entity–Entity Relation Dataset, illustrating text collection from Bangla_MER and Somikoron datasets, duplicate handling, entity extraction, equation analysis, and manual relation labeling.
  • Figure 2: Donut chart showing the percentage distribution of mathematical relation classes in the constructed dataset, highlighting the dominance of subtle Square Root operations and the balanced representation of Addition, Subtraction, Multiplication, and Division.
  • Figure 3: Illustration of the data preprocessing workflow, demonstrating raw text normalization through character cleaning, stop-word removal, lemmatization, and stemming to generate model-ready mathematical text.
  • Figure 4: Proposed BERT Model
  • Figure 5: The working process of SHAP
  • ...and 7 more figures