Unlocking Structured Thinking in Language Models with Cognitive Prompting
Oliver Kramer, Jill Baumann
TL;DR
The paper addresses the challenge of multi-step reasoning in large language models by introducing cognitive prompting (CP), which structures problem solving into a sequence of cognitive operations (COPs) inspired by cognitive psychology. It formalizes eight COPs (plus domain-specific adaptations) and presents three CP variants: deterministic, self-adaptive, and hybrid with few-shot CoT, showing that CP improves arithmetic reasoning on GSM8K across mid-size and large LLMs, with the hybrid approach often delivering the strongest gains. The contributions include a formal COP framework, domain adaptations for arithmetic, and strong empirical evidence that CP enhances structured thinking without additional training, suggesting broad applicability and interpretability benefits. The work highlights the potential of CP to generalize reasoning templates across tasks and opens avenues for applying structured thinking to diverse domains beyond arithmetic.
Abstract
We propose cognitive prompting as a novel approach to guide problem-solving in large language models (LLMs) through structured, human-like cognitive operations, such as goal clarification, decomposition, filtering, abstraction, and pattern recognition. By employing systematic, step-by-step reasoning, cognitive prompting enables LLMs to tackle complex, multi-step tasks more efficiently. We introduce three variants: a deterministic sequence of cognitive operations, a self-adaptive variant in which the LLM dynamically selects the sequence of cognitive operations, and a hybrid variant that uses generated correct solutions as few-shot chain-of-thought prompts. Experiments with LLaMA, Gemma~2, and Qwen models in each two sizes on the arithmetic reasoning benchmark GSM8K demonstrate that cognitive prompting significantly improves performance compared to standard question answering.