Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Look Before You Leap: Problem Elaboration Prompting Improves Mathematical Reasoning in Large Language Models

Haoran Liao, Jidong Tian, Shaohua Hu, Hao He, Yaohui Jin

TL;DR

This study proposes a new approach named Problem Elaboration Prompting (PEP) to enhance the mathematical capacities of LLMs by decomposing and elucidates the problem context before reasoning, therefore enhancing the context modeling and parsing efficiency.

Abstract

Large language models (LLMs) still grapple with complex tasks like mathematical reasoning. Despite significant efforts invested in improving prefix prompts or reasoning process, the crucial role of problem context might have been neglected. Accurate recognition of inputs is fundamental for solving mathematical tasks, as ill-formed problems could potentially mislead LLM's reasoning. In this study, we propose a new approach named Problem Elaboration Prompting (PEP) to enhance the mathematical capacities of LLMs. Specifically, PEP decomposes and elucidates the problem context before reasoning, therefore enhancing the context modeling and parsing efficiency. Experiments across datasets and models demonstrate promising performances: (1) PEP demonstrates an overall enhancement in various mathematical tasks. For instance, with the GPT-3.5 model, PEP exhibits improvements of 9.93% and 8.80% on GSM8k through greedy decoding and self-consistency, respectively. (2) PEP can be easily implemented and integrated with other prompting methods. (3) PEP shows particular strength in handling distraction problems.

Look Before You Leap: Problem Elaboration Prompting Improves Mathematical Reasoning in Large Language Models

TL;DR

This study proposes a new approach named Problem Elaboration Prompting (PEP) to enhance the mathematical capacities of LLMs by decomposing and elucidates the problem context before reasoning, therefore enhancing the context modeling and parsing efficiency.

Abstract

Large language models (LLMs) still grapple with complex tasks like mathematical reasoning. Despite significant efforts invested in improving prefix prompts or reasoning process, the crucial role of problem context might have been neglected. Accurate recognition of inputs is fundamental for solving mathematical tasks, as ill-formed problems could potentially mislead LLM's reasoning. In this study, we propose a new approach named Problem Elaboration Prompting (PEP) to enhance the mathematical capacities of LLMs. Specifically, PEP decomposes and elucidates the problem context before reasoning, therefore enhancing the context modeling and parsing efficiency. Experiments across datasets and models demonstrate promising performances: (1) PEP demonstrates an overall enhancement in various mathematical tasks. For instance, with the GPT-3.5 model, PEP exhibits improvements of 9.93% and 8.80% on GSM8k through greedy decoding and self-consistency, respectively. (2) PEP can be easily implemented and integrated with other prompting methods. (3) PEP shows particular strength in handling distraction problems.
Paper Structure (38 sections, 3 figures, 10 tables)

This paper contains 38 sections, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Emergent Abilities and Prompting
  4. Improving CoT Reasoning
  5. Problem Elaboration Prompting
  6. Method Formulation
  7. Designing Principles
  8. Prompts Generation
  9. Experiment
  10. Setup
  11. Datasets.
  12. Baselines & Prompts.
  13. Language Models & Decoding
  14. Main Results
  15. PEP performs well in a variety of situations.
  16. ...and 23 more sections

Figures (3)

  • Figure 1: We proposed Problem Elaboration Prompting (PEP) for enhancing problem context, thereby improving subsequent reasoning. As depicted in the example, PEP decouples spurious relationships and refines statements, preventing downstream distraction errors.
  • Figure 2: An overview of the proposed PEP and other problem-related methods. Rather than creating sub-questions or plans to guide subsequent reasoning, PEP focuses on clarifying and enriching the problem context, i.e., PEP can be integrated with these methods.
  • Figure 3: Breakdown accuracies w.r.t. irrelevant sentence factors (T: Topic, RO: Role Overlap, NR: Num. Range). Lower accuracy suggests the model is more sensitive to that factor.