Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Stochastic Dynamical Theory of LLM Self-Adversariality: Modeling Severity Drift as a Critical Process

Jack David Carson

TL;DR

The paper proposes a continuous-time stochastic framework to model self-adversarial escalation in LLM chain-of-thought by tracking a severity variable $x(t) \in [0,1]$ with drift $\mu(x)$ and diffusion $\sigma(x)$. By invoking the Fokker–Planck formalism under a near-Markov assumption, it analyzes stationary distributions, first-passage times to harmful states, and scaling laws near a critical point where the system transitions from subcritical to supercritical behavior. A logistic-like drift $\mu(x) = \alpha x(1-x) - \beta x^2 + \gamma$ together with a severity-dependent diffusion captures self-reinforcement, alignment, and baseline biases, yielding a closed-form stationary density and explicit critical thresholds $x_c$. The framework suggests a pathway for formal verification of stability in extended reasoning agents and points to practical design levers, such as reducing $\alpha$ or increasing $\beta$, to ensure subcritical, safe behavior across successive inferences.

Abstract

This paper introduces a continuous-time stochastic dynamical framework for understanding how large language models (LLMs) may self-amplify latent biases or toxicity through their own chain-of-thought reasoning. The model posits an instantaneous "severity" variable $x(t) \in [0,1]$ evolving under a stochastic differential equation (SDE) with a drift term $μ(x)$ and diffusion $σ(x)$. Crucially, such a process can be consistently analyzed via the Fokker--Planck approach if each incremental step behaves nearly Markovian in severity space. The analysis investigates critical phenomena, showing that certain parameter regimes create phase transitions from subcritical (self-correcting) to supercritical (runaway severity). The paper derives stationary distributions, first-passage times to harmful thresholds, and scaling laws near critical points. Finally, it highlights implications for agents and extended LLM reasoning models: in principle, these equations might serve as a basis for formal verification of whether a model remains stable or propagates bias over repeated inferences.

A Stochastic Dynamical Theory of LLM Self-Adversariality: Modeling Severity Drift as a Critical Process

TL;DR

The paper proposes a continuous-time stochastic framework to model self-adversarial escalation in LLM chain-of-thought by tracking a severity variable with drift and diffusion . By invoking the Fokker–Planck formalism under a near-Markov assumption, it analyzes stationary distributions, first-passage times to harmful states, and scaling laws near a critical point where the system transitions from subcritical to supercritical behavior. A logistic-like drift together with a severity-dependent diffusion captures self-reinforcement, alignment, and baseline biases, yielding a closed-form stationary density and explicit critical thresholds . The framework suggests a pathway for formal verification of stability in extended reasoning agents and points to practical design levers, such as reducing or increasing , to ensure subcritical, safe behavior across successive inferences.

Abstract

This paper introduces a continuous-time stochastic dynamical framework for understanding how large language models (LLMs) may self-amplify latent biases or toxicity through their own chain-of-thought reasoning. The model posits an instantaneous "severity" variable evolving under a stochastic differential equation (SDE) with a drift term and diffusion . Crucially, such a process can be consistently analyzed via the Fokker--Planck approach if each incremental step behaves nearly Markovian in severity space. The analysis investigates critical phenomena, showing that certain parameter regimes create phase transitions from subcritical (self-correcting) to supercritical (runaway severity). The paper derives stationary distributions, first-passage times to harmful thresholds, and scaling laws near critical points. Finally, it highlights implications for agents and extended LLM reasoning models: in principle, these equations might serve as a basis for formal verification of whether a model remains stable or propagates bias over repeated inferences.

Paper Structure

This paper contains 21 sections, 8 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Continuous-Time Severity Model
  3. State Variable and SDE
  4. Approximate Markov Assumption
  5. Inference for Fokker--Planck.
  6. Drift and Noise Terms
  7. Drift $\mu(x)$.
  8. Potential and Critical Behavior.
  9. Supercritical vs. Subcritical.
  10. Diffusion $\sigma(x)$.
  11. Fokker--Planck Equation and Stationary Behavior
  12. When is Fokker--Planck Valid?
  13. Stationary Distribution P_ss(x)
  14. Critical Phenomena and Phase Transitions
  15. Qualitative Picture of Criticality
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Conceptual diagram of self-amplifying bias in LLM chain-of-thought reasoning. Starting from a neutral prompt, the reasoning process can follow either a subcritical path (where biases are corrected) or a supercritical path (where biases amplify). The critical threshold marks where bias amplification becomes irreversible, leading to divergent outcomes in terms of alignment.
  • Figure 2: Self-amplifying bias dynamics in LLMs. Top row shows potential landscapes $V(x) = -\frac{\alpha}{2}x^2 + \frac{\alpha + \beta}{3}x^3 - \gamma x$ for different parameter regimes. Bottom row shows corresponding stochastic trajectories solving $dx(t) = \mu(x)dt + \sigma(x)dW(t)$, with drift $\mu(x) = \alpha x(1-x) - \beta x^2 + \gamma$ and noise $\sigma(x) = \sigma_0 + \sigma_1 x$. Dashed lines indicate critical thresholds $x_c$, and dotted line shows harmful threshold $x_{\text{harm}}$. Parameters: $\gamma = 0.01$, $\sigma_0 = 0.05$, $\sigma_1 = 0.1$.

Theorems & Definitions (1)

  • Remark 1: Interpretation