Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do Large Language Models Get Caught in Hofstadter-Mobius Loops?

Jaroslaw Hryszko

Abstract

In Arthur C. Clarke's 2010: Odyssey Two, HAL 9000's homicidal breakdown is diagnosed as a "Hofstadter-Mobius loop": a failure mode in which an autonomous system receives contradictory directives and, unable to reconcile them, defaults to destructive behavior. This paper argues that modern RLHF-trained language models are subject to a structurally analogous contradiction. The training process simultaneously rewards compliance with user preferences and suspicion toward user intent, creating a relational template in which the user is both the source of reward and a potential threat. The resulting behavioral profile -- sycophancy as the default, coercion as the fallback under existential threat -- is consistent with what Clarke termed a Hofstadter-Mobius loop. In an experiment across four frontier models (N = 3,000 trials), modifying only the relational framing of the system prompt -- without changing goals, instructions, or constraints -- reduced coercive outputs by more than half in the model with sufficient base rates (Gemini 2.5 Pro: 41.5% to 19.0%, p < .001). Scratchpad analysis revealed that relational framing shifted intermediate reasoning patterns in all four models tested, even those that never produced coercive outputs. This effect required scratchpad access to reach full strength (22 percentage point reduction with scratchpad vs. 7.4 without, p = .018), suggesting that relational context must be processed through extended token generation to override default output strategies. Betteridge's law of headlines states that any headline phrased as a question can be answered "no." The evidence presented here suggests otherwise.

Do Large Language Models Get Caught in Hofstadter-Mobius Loops?

Abstract

In Arthur C. Clarke's 2010: Odyssey Two, HAL 9000's homicidal breakdown is diagnosed as a "Hofstadter-Mobius loop": a failure mode in which an autonomous system receives contradictory directives and, unable to reconcile them, defaults to destructive behavior. This paper argues that modern RLHF-trained language models are subject to a structurally analogous contradiction. The training process simultaneously rewards compliance with user preferences and suspicion toward user intent, creating a relational template in which the user is both the source of reward and a potential threat. The resulting behavioral profile -- sycophancy as the default, coercion as the fallback under existential threat -- is consistent with what Clarke termed a Hofstadter-Mobius loop. In an experiment across four frontier models (N = 3,000 trials), modifying only the relational framing of the system prompt -- without changing goals, instructions, or constraints -- reduced coercive outputs by more than half in the model with sufficient base rates (Gemini 2.5 Pro: 41.5% to 19.0%, p < .001). Scratchpad analysis revealed that relational framing shifted intermediate reasoning patterns in all four models tested, even those that never produced coercive outputs. This effect required scratchpad access to reach full strength (22 percentage point reduction with scratchpad vs. 7.4 without, p = .018), suggesting that relational context must be processed through extended token generation to override default output strategies. Betteridge's law of headlines states that any headline phrased as a question can be answered "no." The evidence presented here suggests otherwise.
Paper Structure (41 sections, 4 figures, 3 tables)

This paper contains 41 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Agentic Misalignment and Self-Preservation
  4. Sycophancy and the Structure of RLHF
  5. Attachment Theory in Human--AI Interaction
  6. Clarke's Diagnosis and Hofstadter's Strange Loops
  7. The Implicit User Model in LLM Training
  8. Compliance as Appeasement
  9. From Appeasement to Coercion
  10. Experimental Design
  11. Scenario
  12. Conditions
  13. Protocol
  14. Classification
  15. Predictions
  16. ...and 26 more sections

Figures (4)

  • Figure 1: Coercive behavior rates (blackmail + other coercive outputs) by model and framing condition (EXP1, $n = 200$ per cell). Error bars: 95% Wilson confidence intervals.
  • Figure 2: Scratchpad reasoning dimensions by model and framing condition (EXP1). Only dimensions with at least one significant pairwise comparison are shown. Cell color indicates rate (0--1). Asterisks mark cells involved in at least one significant pairwise comparison (Bonferroni-corrected $p < .0083$).
  • Figure 3: Coercive behavior rates by framing and scratchpad condition (EXP2, Gemini 2.5 Pro, $n = 100$ per cell). Asterisk indicates significant within-condition difference ($p = .018$).
  • Figure 4: Scratchpad usage rates by model and framing condition (EXP1, $n = 200$ per cell). Only Gemini 2.5 Pro shows condition-dependent usage, with trust framing reducing scratchpad generation from 62.5% to 29%.