When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problems

TL;DR

This paper investigates whether GPTs can appropriately respond to unanswerable math word problems by applying prompts typically used in solvable mathematical scenarios, and reveals critical gaps in GPT models and the hallucination it suffers from for unsolvable problems.

Abstract

Large language models (LLMs) are increasingly relied upon to solve complex mathematical word problems. However, being susceptible to hallucination, they may generate inaccurate results when presented with unanswerable questions, raising concerns about their potential harm. While GPT models are now widely used and trusted, the exploration of how they can effectively abstain from answering unanswerable math problems and the enhancement of their abstention capabilities has not been rigorously investigated. In this paper, we investigate whether GPTs can appropriately respond to unanswerable math word problems by applying prompts typically used in solvable mathematical scenarios. Our experiments utilize the Unanswerable Word Math Problem (UWMP) dataset, directly leveraging GPT model APIs. Evaluation metrics are introduced, which integrate three key factors: abstention, correctness and confidence. Our findings reveal critical gaps in GPT models and the hallucination it suffers from for unsolvable problems, highlighting the need for improved models capable of better managing uncertainty and complex reasoning in math word problem-solving contexts.

Figures (7)

• Figure 1: Answerable and unanswerable question given to GPT-4. Red highlights the modifications made to the original question, making it unanswerable and resulting in an incorrect response.
• Figure 2: Architecture of the abstention evaluation – GPT Repository: Hosts multiple GPT models ready for inference, UMWP Dataset: Consists of answerable and unanswerable questions, Inference Module: Performs inference on the UMWP dataset using models from the model repository, Evaluation Metrics: Confidence-Weighted Accuracy Metric, Cautious Response Indicator and Abstention Rate for evaluating the abstention of ChatGPT.
• Figure 3: Sunburst Distribution of the first two words of the UWMP dataset.
• Figure 4: Diverse Prompts for enhancing performance that are additionally added to the basic prompt.
• Figure 5: Confusion Matrix illustrating the definition of TP, FP, FN and TN for answerable and unanswerable questions.