Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities

Hans Hergen Lehmann, Jae Hee Lee, Steven Schockaert, Stefan Wermter

TL;DR

The paper investigates whether large language models truly understand numerical attributes or rely on superficial cues when ranking entities. By testing 10 Wikidata-derived attributes with balanced, orthogonal prompt designs across model sizes, it shows that larger models predict pairwise rankings more accurately but not because they know the numbers more accurately; instead they strategically favor numerical evidence when reliable. It identifies dominant biases—prompt position, popularity, and co-occurrence—and demonstrates that a simple meta-predictor based on surface cues can explain much of the pairwise decisions, especially for smaller models. Chain-of-thought prompting partially aligns predictions with numerical knowledge, improving consistency but not fully eliminating heuristic use, suggesting that scale enables more principled strategy selection along with knowledge growth.

Abstract

Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. We investigate this question through entity comparison tasks by asking models to compare entities along numerical attributes (e.g., ``Which river is longer, the Danube or the Nile?''), which offer clear ground truth for systematic analysis. Despite having sufficient numerical knowledge to answer correctly, LLMs frequently make predictions that contradict this knowledge. We identify three heuristic biases that strongly influence model predictions: entity popularity, mention order, and semantic co-occurrence. For smaller models, a simple logistic regression using only these surface cues predicts model choices more accurately than the model's own numerical predictions, suggesting heuristics largely override principled reasoning. Crucially, we find that larger models (32B parameters) selectively rely on numerical knowledge when it is more reliable, while smaller models (7--8B parameters) show no such discrimination, which explains why larger models outperform smaller ones even when the smaller models possess more accurate knowledge. Chain-of-thought prompting steers all models towards using the numerical features across all model sizes.

Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities

TL;DR

The paper investigates whether large language models truly understand numerical attributes or rely on superficial cues when ranking entities. By testing 10 Wikidata-derived attributes with balanced, orthogonal prompt designs across model sizes, it shows that larger models predict pairwise rankings more accurately but not because they know the numbers more accurately; instead they strategically favor numerical evidence when reliable. It identifies dominant biases—prompt position, popularity, and co-occurrence—and demonstrates that a simple meta-predictor based on surface cues can explain much of the pairwise decisions, especially for smaller models. Chain-of-thought prompting partially aligns predictions with numerical knowledge, improving consistency but not fully eliminating heuristic use, suggesting that scale enables more principled strategy selection along with knowledge growth.

Abstract

Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. We investigate this question through entity comparison tasks by asking models to compare entities along numerical attributes (e.g., ``Which river is longer, the Danube or the Nile?''), which offer clear ground truth for systematic analysis. Despite having sufficient numerical knowledge to answer correctly, LLMs frequently make predictions that contradict this knowledge. We identify three heuristic biases that strongly influence model predictions: entity popularity, mention order, and semantic co-occurrence. For smaller models, a simple logistic regression using only these surface cues predicts model choices more accurately than the model's own numerical predictions, suggesting heuristics largely override principled reasoning. Crucially, we find that larger models (32B parameters) selectively rely on numerical knowledge when it is more reliable, while smaller models (7--8B parameters) show no such discrimination, which explains why larger models outperform smaller ones even when the smaller models possess more accurate knowledge. Chain-of-thought prompting steers all models towards using the numerical features across all model sizes.
Paper Structure (44 sections, 7 equations, 14 figures, 18 tables)

This paper contains 44 sections, 7 equations, 14 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Experimental Setup
  3. Datasets.
  4. Prompting Strategy.
  5. Evaluation Metrics.
  6. Models.
  7. Do LLMs Use Numerical Attributes for Pairwise Comparisons?
  8. How Susceptible Are Pairwise Predictions to Biases?
  9. Heuristic Cues.
  10. Experimental Setup.
  11. Results.
  12. Can LLM Predictions Be Explained?
  13. Fine-grained Analysis
  14. When Do Models Rely on Surface Cues?
  15. How does CoT Affect Predictions?
  16. ...and 29 more sections

Figures (14)

  • Figure 1: Overall performance in terms of pairwise accuracy, internal consistency, and numerical accuracy (mean and standard deviation)
  • Figure 2: Risk ratios (log scale) for each feature, with 95% confidence intervals from bootstrapping over prompt templates and datasets. See \ref{['fig:bos_detailed']} in \ref{['app:sec:bos_detailed']} for per-dataset, per-model results.
  • Figure 3: For each language model, we report the mean improvement and standard deviation of a logistic-regression meta-predictor that relies solely on three surface cues, relative to a strategy that follows the model's extracted numerical values. Positive values indicate that the bias-based predictor anticipates the model's pairwise choice more accurately than the model's own numbers.
  • Figure 4: Proportion of the four cases for different models, aggregated over all datasets. For each of the four cases, the upper (darker) segment marks correct predictions, and the lower (lighter) segment marks errors with respect to the ground truth.
  • Figure 5: Signed Cohen's $d$ for contrasting Case 1 vs. Case 3. Green means the feature value tends to be larger in Case 1 (numerical knowledge), red means that the value tends to be larger in Case 3 (surface cues). Cell labels show the mean effect.
  • ...and 9 more figures