Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty

Haomiaomiao Wang, Tomás E Ward, Lili Zhang

Abstract

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.

Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty

Abstract

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.

Paper Structure

This paper contains 36 sections, 12 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Models and decoding settings
  4. Reversal-learning task and experimental design
  5. Latent states.
  6. Switch trigger.
  7. Fixed versus random schedules.
  8. Prompting protocol and response validation
  9. Prompt-format controls and label selection
  10. Behavioural measures
  11. Reversal-sensitive measures.
  12. Perseveration length.
  13. Post-reversal regret.
  14. Overall task performance.
  15. Computational models
  16. ...and 21 more sections

Figures (2)

  • Figure 1: Random schedule: post-reversal adoption of the target option and event-aligned choice proportions around reversals.
  • Figure 2: Posterior group-level parameter densities under Dual RL and Dual RL-$\kappa$DU (left vs. right rows) for fixed and random schedules (bottom vs. top rows). Panels show gain and loss learning rates ($\eta_{\mathrm{pos}}, \eta_{\mathrm{neg}}$), inverse temperature ($\beta$), and (for $\kappa$DU) the counterfactual updating parameter ($\kappa$).