Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram

Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu

TL;DR

This work tackles geometry problem solving (GPS) by introducing PGPSNet, a diagram-text fusion neural solver that converts geometry diagrams into textual structural and semantic clauses and generates an interpretable solution program. It couples a diagram encoder, a clause-aware text encoder, and a self-limited decoder, enhanced by structural/semantic pre-training and five data-augmentation strategies. The authors also provide PGPS9K, a large-scale GPS dataset with fine-grained diagram annotations and an interpretable solution program, enabling rigorous evaluation against state-of-the-art neural and symbolic solvers. Experimental results on Geometry3K and PGPS9K show that PGPSNet substantially improves neural GPS performance and narrows the gap with symbolic approaches, highlighting the potential of clause-based diagram representations for multi-modal geometric reasoning.

Abstract

Geometry problem solving (GPS) is a high-level mathematical reasoning requiring the capacities of multi-modal fusion and geometric knowledge application. Recently, neural solvers have shown great potential in GPS but still be short in diagram presentation and modal fusion. In this work, we convert diagrams into basic textual clauses to describe diagram features effectively, and propose a new neural solver called PGPSNet to fuse multi-modal information efficiently. Combining structural and semantic pre-training, data augmentation and self-limited decoding, PGPSNet is endowed with rich knowledge of geometry theorems and geometric representation, and therefore promotes geometric understanding and reasoning. In addition, to facilitate the research of GPS, we build a new large-scale and fine-annotated GPS dataset named PGPS9K, labeled with both fine-grained diagram annotation and interpretable solution program. Experiments on PGPS9K and an existing dataset Geometry3K validate the superiority of our method over the state-of-the-art neural solvers. Our code, dataset and appendix material are available at \url{https://github.com/mingliangzhang2018/PGPS}.

A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram

TL;DR

This work tackles geometry problem solving (GPS) by introducing PGPSNet, a diagram-text fusion neural solver that converts geometry diagrams into textual structural and semantic clauses and generates an interpretable solution program. It couples a diagram encoder, a clause-aware text encoder, and a self-limited decoder, enhanced by structural/semantic pre-training and five data-augmentation strategies. The authors also provide PGPS9K, a large-scale GPS dataset with fine-grained diagram annotations and an interpretable solution program, enabling rigorous evaluation against state-of-the-art neural and symbolic solvers. Experimental results on Geometry3K and PGPS9K show that PGPSNet substantially improves neural GPS performance and narrows the gap with symbolic approaches, highlighting the potential of clause-based diagram representations for multi-modal geometric reasoning.

Abstract

Geometry problem solving (GPS) is a high-level mathematical reasoning requiring the capacities of multi-modal fusion and geometric knowledge application. Recently, neural solvers have shown great potential in GPS but still be short in diagram presentation and modal fusion. In this work, we convert diagrams into basic textual clauses to describe diagram features effectively, and propose a new neural solver called PGPSNet to fuse multi-modal information efficiently. Combining structural and semantic pre-training, data augmentation and self-limited decoding, PGPSNet is endowed with rich knowledge of geometry theorems and geometric representation, and therefore promotes geometric understanding and reasoning. In addition, to facilitate the research of GPS, we build a new large-scale and fine-annotated GPS dataset named PGPS9K, labeled with both fine-grained diagram annotation and interpretable solution program. Experiments on PGPS9K and an existing dataset Geometry3K validate the superiority of our method over the state-of-the-art neural solvers. Our code, dataset and appendix material are available at \url{https://github.com/mingliangzhang2018/PGPS}.
Paper Structure (23 sections, 3 equations, 6 figures, 5 tables)

This paper contains 23 sections, 3 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-modal Reasoning
  4. Geometry Problem Solving
  5. Pre-trained Language Model for Mathematical Reasoning
  6. PGPS9K Dataset
  7. Collection and Description
  8. Annotation Form
  9. PGPSNet Model
  10. Overall Framework
  11. Structural and Semantic Pre-training
  12. Encoder and Decoder
  13. Data Augmentation
  14. Experiment
  15. Setup
  16. ...and 8 more sections

Figures (6)

  • Figure 1: Overview of PGPSNet solver. PGPSNet is a multi-modal learning framework whose modal inputs contain not only the diagram and textual problem, but also the textual clauses parsed from diagram. It generates the theorem-based interpretable solution program to solve geometry problem.
  • Figure 2: Example presentation of PGPS9K dataset.
  • Figure 3: Annotation of solution program and its interpretability.
  • Figure 4: Pipeline of structural and semantic pre-training. [M] denotes the mask token. Class tags of [G], [N], [ARG], [P], [ANGID] represent tokens of general, variable, argument, point and angle ID, respectively. Section tags of [S], [C], [T] refer to tokens of structure, condition and target, respectively.
  • Figure 5: Case studies on PGPS9K.
  • ...and 1 more figures