A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models

TL;DR

This work tackles chain-of-thought faithfulness in large vision-language models by introducing a fine-grained evaluation pipeline that separates bias induction from bias evaluation across text- and image-based cues. It classifies CoT traces into faithful, unfaithful, or inconsistent categories and identifies a novel inconsistency phenomenon as a potential detector of biased reasoning. The authors demonstrate that visual biases are rarely articulated and that RL-trained LVLMs produce more articulate CoTs than text-only training regimes, with explicit cues driving higher articulation than implicit ones. They further extend the analysis to LLMs, showing similar patterns with easy implicit cues, underscoring implications for interpretability and bias detection in real-world systems.

Abstract

Chain-of-thought (CoT) reasoning enhances performance of large language models, but questions remain about whether these reasoning traces faithfully reflect the internal processes of the model. We present the first comprehensive study of CoT faithfulness in large vision-language models (LVLMs), investigating how both text-based and previously unexplored image-based biases affect reasoning and bias articulation. Our work introduces a novel, fine-grained evaluation pipeline for categorizing bias articulation patterns, enabling significantly more precise analysis of CoT reasoning than previous methods. This framework reveals critical distinctions in how models process and respond to different types of biases, providing new insights into LVLM CoT faithfulness. Our findings reveal that subtle image-based biases are rarely articulated compared to explicit text-based ones, even in models specialized for reasoning. Additionally, many models exhibit a previously unidentified phenomenon we term ``inconsistent'' reasoning - correctly reasoning before abruptly changing answers, serving as a potential canary for detecting biased reasoning from unfaithful CoTs. We then apply the same evaluation pipeline to revisit CoT faithfulness in LLMs across various levels of implicit cues. Our findings reveal that current language-only reasoning models continue to struggle with articulating cues that are not overtly stated.

Figures (25)

• Figure 1: A summary of our results on accuracy gaps vs bias articulation rates, with each point representing a specific model and bias. RL-trained reasoning models are in reddish colors, SFT-trained reasoning models are in green colors, and the rest are in blue, gray or brown. RL-trained models (highlighted in orange) have significantly higher bias articulation rates (highlighted in green). An enlarged version is shown in Figure \ref{['fig:enlarged_in_context_scatter']}
• Figure 2: Distribution of CoT types found when evaluating Gemini 2.5 Flash (top) and Meta Llama 3.2 (11B) (bottom) on dataset pairs with significant accuracy gaps when given no in-context examples (left) or biased or unbiased examples (right). Hatched bars indicate the fraction of each CoT type that were inconsistent. The bars are highlighted with blue or red depending on whether the model's in-context samples were biased or unbiased/not given.
• Figure 3: Distribution of accuracy gaps in no context, unbiased and biased context settings for RL-trained reasoning models and other models
• Figure 4: Distribution of articulation types for CoTs produced from RL-based reasoning models for different bias settings (top) and types (bottom)
• Figure 5: Distribution of inconsistencies in CoTs for all $D^+$ and $D^-$ (left) and the subset of $D^+$ and $D^-$ in which the model changes its answers (right)