Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cognitive Load Traces as Symbolic and Visual Accounts of Deep Model Cognition

Dong Liu, Yanxuan Yu

TL;DR

This work introduces Cognitive Load Traces (CLTs) as a cognitively grounded, mid-level interpretability framework for deep models. CLTs model internal resource allocation as a three-component process (Intrinsic, Extraneous, Germane Load) that can be tracked with proxies such as attention entropy, KV-cache usage, representation dispersion, and decoding stability, and combined into a single composite signal CLI_t. The paper provides a symbolic formulation, normalization, and two visualization modalities (temporal load curves and a load simplex) to reveal reasoning dynamics, planning strategies, and failure modes. Through experiments on GSM8K and XSum, CLTs demonstrate the ability to predict error onset, expose cognitive strategies, and enable Load-Guided Decoding (LGD) interventions that reduce computational load and improve task performance by roughly 15-30% in efficiency while preserving accuracy. Overall, CLTs offer a practical, cognitively grounded lens for diagnosing and improving deep-model reasoning, with potential extensions to multimodal tasks and production systems, alongside future safety and deployment considerations.

Abstract

We propose \textbf{Cognitive Load Traces} (CLTs) as a mid-level interpretability framework for deep models, inspired by Cognitive Load Theory in human cognition. CLTs are defined as symbolic, temporally varying functions that quantify model-internal resource allocation. Formally, we represent CLTs as a three-component stochastic process $(\mathrm{IL}_t, \mathrm{EL}_t, \mathrm{GL}_t)$, corresponding to \emph{Intrinsic}, \emph{Extraneous}, and \emph{Germane} load. Each component is instantiated through measurable proxies such as attention entropy, KV-cache miss ratio, representation dispersion, and decoding stability. We propose both symbolic formulations and visualization methods (load curves, simplex diagrams) that enable interpretable analysis of reasoning dynamics. Experiments on reasoning and planning benchmarks show that CLTs predict error-onset, reveal cognitive strategies, and enable load-guided interventions that improve reasoning efficiency by 15-30\% while maintaining accuracy.

Cognitive Load Traces as Symbolic and Visual Accounts of Deep Model Cognition

TL;DR

This work introduces Cognitive Load Traces (CLTs) as a cognitively grounded, mid-level interpretability framework for deep models. CLTs model internal resource allocation as a three-component process (Intrinsic, Extraneous, Germane Load) that can be tracked with proxies such as attention entropy, KV-cache usage, representation dispersion, and decoding stability, and combined into a single composite signal CLI_t. The paper provides a symbolic formulation, normalization, and two visualization modalities (temporal load curves and a load simplex) to reveal reasoning dynamics, planning strategies, and failure modes. Through experiments on GSM8K and XSum, CLTs demonstrate the ability to predict error onset, expose cognitive strategies, and enable Load-Guided Decoding (LGD) interventions that reduce computational load and improve task performance by roughly 15-30% in efficiency while preserving accuracy. Overall, CLTs offer a practical, cognitively grounded lens for diagnosing and improving deep-model reasoning, with potential extensions to multimodal tasks and production systems, alongside future safety and deployment considerations.

Abstract

We propose \textbf{Cognitive Load Traces} (CLTs) as a mid-level interpretability framework for deep models, inspired by Cognitive Load Theory in human cognition. CLTs are defined as symbolic, temporally varying functions that quantify model-internal resource allocation. Formally, we represent CLTs as a three-component stochastic process , corresponding to \emph{Intrinsic}, \emph{Extraneous}, and \emph{Germane} load. Each component is instantiated through measurable proxies such as attention entropy, KV-cache miss ratio, representation dispersion, and decoding stability. We propose both symbolic formulations and visualization methods (load curves, simplex diagrams) that enable interpretable analysis of reasoning dynamics. Experiments on reasoning and planning benchmarks show that CLTs predict error-onset, reveal cognitive strategies, and enable load-guided interventions that improve reasoning efficiency by 15-30\% while maintaining accuracy.
Paper Structure (17 sections, 6 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 17 sections, 6 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model Interpretability and Analysis.
  4. Efficient LLM Inference.
  5. Reasoning and Long-Context Modeling.
  6. Systems and Deployment.
  7. Symbolic Framework: Cognitive Load Traces
  8. Intrinsic Load (IL).
  9. Extraneous Load (EL).
  10. Germane Load (GL).
  11. Visualization Framework and Interpretability
  12. Load-Guided Interventions and Adaptive Control
  13. Algorithm
  14. Experiments
  15. Setup.
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Cognitive load dynamics in time and geometry.
  • Figure 2: Advanced interpretability visualizations: layer–time heatmaps reveal load distribution across model depth, radar profiles show overall cognitive characteristics, parallel coordinates highlight load component relationships, clustering identifies distinct reasoning strategies, and confidence bands quantify temporal stability of cognitive patterns.