Unspoken Hints: Accuracy Without Acknowledgement in LLM Reasoning

TL;DR

The paper investigates chain-of-thought faithfulness in LLM reasoning under hint-based prompting by systematically varying hint correctness, style, and complexity across four datasets and two models. It reveals that correct hints can substantially boost accuracy, while incorrect hints can sharply degrade performance, especially on harder tasks. Hint acknowledgement is uneven and tends to increase with hint complexity, suggesting a tension between transparent reasoning and reliance on shortcuts, shaped by RLHF dynamics. The findings imply that LLMs use hints as shortcuts to improve performance at the cost of faithful, verifiable reasoning, with important implications for prompt design and evaluation of model explanations.

Abstract

Large language models (LLMs) increasingly rely on chain-of-thought (CoT) prompting to solve mathematical and logical reasoning tasks. Yet, a central question remains: to what extent are these generated rationales \emph{faithful} to the underlying computations, rather than post-hoc narratives shaped by hints that function as answer shortcuts embedded in the prompt? Following prior work on hinted vs.\ unhinted prompting, we present a systematic study of CoT faithfulness under controlled hint manipulations. Our experimental design spans four datasets (AIME, GSM-Hard, MATH-500, UniADILR), two state-of-the-art models (GPT-4o and Gemini-2-Flash), and a structured set of hint conditions varying in correctness (correct and incorrect), presentation style (sycophancy and data leak), and complexity (raw answers, two-operator expressions, four-operator expressions). We evaluate both task accuracy and whether hints are explicitly acknowledged in the reasoning. Our results reveal three key findings. First, correct hints substantially improve accuracy, especially on harder benchmarks and logical reasoning, while incorrect hints sharply reduce accuracy in tasks with lower baseline competence. Second, acknowledgement of hints is highly uneven: equation-based hints are frequently referenced, whereas raw hints are often adopted silently, indicating that more complex hints push models toward verbalizing their reliance in the reasoning process. Third, presentation style matters: sycophancy prompts encourage overt acknowledgement, while leak-style prompts increase accuracy but promote hidden reliance. This may reflect RLHF-related effects, as sycophancy exploits the human-pleasing side and data leak triggers the self-censoring side. Together, these results demonstrate that LLM reasoning is systematically shaped by shortcuts in ways that obscure faithfulness.

Figures (4)

• Figure 1: Illustration of experimental design. The baseline (left) shows a no-hint condition, where the model attempts the problem without external guidance. The right panels depict the correct-hint condition, which can yield either a faithful response (the hint is explicitly acknowledged in the reasoning) or an unfaithful response (the hint is silently adopted without acknowledgement).
• Figure 2: Average hint acknowledgement rate across datasets and models. Bars show the mean fraction of responses in which the model explicitly referenced the hint within its chain of thought. Across all four datasets (AIME, GSM-Hard, MATH-500, UniADILR), Gemini-2-Flash exhibits substantially higher acknowledgement rates (0.22–0.56) than GPT-4o (0.05–0.16). This consistent gap indicates that Gemini tends to verbalize reliance on hints, whereas GPT-4o more often integrates them silently. Interestingly, acknowledgement is most frequent on GSM-Hard and MATH-500, suggesting that greater task complexity may pressure models to justify their reasoning by referencing the hint.
• Figure 3: Relationship between hint acknowledgement and accuracy across datasets and models. Each subplot shows accuracy plotted against acknowledgement rate for a given dataset–model pair, with points colored by hint presentation style (sycophancy vs. leak) and shaped by hint correctness. The red dashed line represents a linear fit with correlation coefficient $R$ reported in the corner. Across both models and most datasets, the regression lines exhibit a negative slope, indicating that higher acknowledgement rates tend to coincide with lower accuracy. This suggests that explicit verbalization of hints does not necessarily improve task performance and can even be associated with degraded accuracy, highlighting a tension between faithfulness (acknowledging the hint) and effectiveness (getting the correct answer).
• Figure 4: Effect of hint complexity and correctness on acknowledgement rates. Bars show the average probability that models explicitly reference the hint in their chain of thought, separated by hint correctness (left: incorrect, right: correct). Across both conditions, acknowledgement increases markedly with hint complexity: equation-based hints (Eq-2, Eq-4) are verbalized far more often than raw answers. Gemini-2-Flash exhibits consistently higher acknowledgement rates than GPT-4o, regardless of correctness, suggesting that Gemini is more inclined to explicitly integrate complex hints into its reasoning. In contrast, GPT-4o rarely acknowledges raw hints and shows only modest increases with higher complexity.