Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems

Jason Starace, Terence Soule

TL;DR

Empirical bounds on behavioral inference are established through controlled experiments at scale, establishing that the bottleneck is entirely located in belief system inference and confirming that the ceiling is information-theoretic rather than data-limited.

Abstract

We establish empirical bounds on behavioral inference through controlled experiments at scale: LLM-based agents assigned one of 36 behavioral profiles (9 belief systems x 4 motivations) generate over 1.5 million behavioral sequences across 17,411 games in grid-world environments, providing ground truth unavailable in human behavioral studies. Rather than asking whether inference has limits, we ask how large those limits are, where they concentrate, and why. A fundamental asymmetry emerges in both magnitude and structure. Motivations achieve 98-100% inference accuracy and recover 97% of available mutual information across all architectures. Belief systems plateau at 24% for LSTMs regardless of capacity, recovering only 30% of available information, a 3.3x asymmetry in information extraction efficiency. Transformer architectures with 9-stage curriculum learning reach 49% alignment accuracy, doubling LSTM performance and demonstrating that the recurrent ceiling is architectural rather than fundamental. Yet even this improvement leaves belief systems correctly classified less than half the time, with per-alignment accuracy ranging from 1% (True Neutral) to 72% (Lawful Evil). Confusion analysis maps the failure structure precisely: a "neutral zone" of behavioral ambiguity extends beyond True Neutral to encompass Good alignments, where prosocial behavior is indistinguishable from rule-following or balance-keeping. Combined motivation and belief inference yields 17.6x improvement over random baseline for full 36-class profile classification, while establishing that the bottleneck is entirely located in belief system inference. Signal enhancement and explanatory queries yield only marginal LSTM gains (+3.8%), confirming that the ceiling is information-theoretic rather than data-limited. These bounds have direct implications for any system relying on behavioral monitoring to infer agent values.

Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems

TL;DR

Empirical bounds on behavioral inference are established through controlled experiments at scale, establishing that the bottleneck is entirely located in belief system inference and confirming that the ceiling is information-theoretic rather than data-limited.

Abstract

We establish empirical bounds on behavioral inference through controlled experiments at scale: LLM-based agents assigned one of 36 behavioral profiles (9 belief systems x 4 motivations) generate over 1.5 million behavioral sequences across 17,411 games in grid-world environments, providing ground truth unavailable in human behavioral studies. Rather than asking whether inference has limits, we ask how large those limits are, where they concentrate, and why. A fundamental asymmetry emerges in both magnitude and structure. Motivations achieve 98-100% inference accuracy and recover 97% of available mutual information across all architectures. Belief systems plateau at 24% for LSTMs regardless of capacity, recovering only 30% of available information, a 3.3x asymmetry in information extraction efficiency. Transformer architectures with 9-stage curriculum learning reach 49% alignment accuracy, doubling LSTM performance and demonstrating that the recurrent ceiling is architectural rather than fundamental. Yet even this improvement leaves belief systems correctly classified less than half the time, with per-alignment accuracy ranging from 1% (True Neutral) to 72% (Lawful Evil). Confusion analysis maps the failure structure precisely: a "neutral zone" of behavioral ambiguity extends beyond True Neutral to encompass Good alignments, where prosocial behavior is indistinguishable from rule-following or balance-keeping. Combined motivation and belief inference yields 17.6x improvement over random baseline for full 36-class profile classification, while establishing that the bottleneck is entirely located in belief system inference. Signal enhancement and explanatory queries yield only marginal LSTM gains (+3.8%), confirming that the ceiling is information-theoretic rather than data-limited. These bounds have direct implications for any system relying on behavioral monitoring to infer agent values.

Paper Structure

This paper contains 51 sections, 3 equations, 3 figures, 16 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experimental Design
  5. Feature Representation
  6. Architecture
  7. Curriculum Learning
  8. Training Protocol
  9. Behavioral Consistency Verification
  10. Results
  11. Recurrent Architecture Ceiling
  12. Curriculum Learning Breakthrough
  13. Feature Ablation
  14. The Neutral Zone Problem
  15. Signal Enhancement Limits
  16. ...and 36 more sections

Figures (3)

  • Figure 1: Player2Vec architecture with curriculum learning. Input features (4,653D) pass through a feature encoder to the curriculum block containing a 6-layer Longformer transformer with local attention, followed by mean pooling and a classification head.
  • Figure 2: Alignment confusion matrix for transformer with curriculum learning. True Neutral (TN) achieves near-zero accuracy; predictions concentrate on moral extremes. Adjacent alignments show systematic confusion, particularly within the neutral zone.
  • Figure 3: Training and validation loss (left) and accuracy (right) across all 9 curriculum stages. Sharp resets mark stage transitions; each stage inherits weights from the previous stage's best checkpoint.