Is your LLM trapped in a Mental Set? Investigative study on how mental sets affect the reasoning capabilities of LLMs

TL;DR

The paper investigates whether mental-set–like cognitive rigidity constrains LLM reasoning when faced with unfamiliar problems. It combines cognitive psychology concepts with LLM evaluation by using a math-equivalence dataset that contrasts complex vs shortcut problems and manipulates problem order and prompting strategies. Key findings indicate that in-context demonstrations improve accuracy but do not reduce the number of reasoning steps, while zero-shot chain-of-thought prompting often increases reasoning depth and steps, revealing a trade-off between accuracy and adaptability. The work highlights the need for evaluation frameworks that capture cognitive flexibility and motivates future extension to vision-language models and broader problem domains to enhance practical adaptability of AI systems.

Abstract

In this paper, we present an investigative study on how Mental Sets influence the reasoning capabilities of LLMs. LLMs have excelled in diverse natural language processing (NLP) tasks, driven by advancements in parameter-efficient fine-tuning (PEFT) and emergent capabilities like in-context learning (ICL). For complex reasoning tasks, selecting the right model for PEFT or ICL is critical, often relying on scores on benchmarks such as MMLU, MATH, and GSM8K. However, current evaluation methods, based on metrics like F1 Score or reasoning chain assessments by larger models, overlook a key dimension: adaptability to unfamiliar situations and overcoming entrenched thinking patterns. In cognitive psychology, Mental Set refers to the tendency to persist with previously successful strategies, even when they become inefficient - a challenge for problem solving and reasoning. We compare the performance of LLM models like Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct and GPT-4o in the presence of mental sets. To the best of our knowledge, this is the first study to integrate cognitive psychology concepts into the evaluation of LLMs for complex reasoning tasks, providing deeper insights into their adaptability and problem-solving efficacy.