Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models

TL;DR

The paper investigates whether large language models exhibit belief-guided agency and meta-cognitive monitoring by operationalizing HOT-3 into measurable mechanisms. It defines beliefs as latent representations, actions as final outputs, and meta-cognition as internal monitoring, introducing Belief Dominance ($BD$) and Belief Dominance Difference ($BDDiff$) quantified via Patchscopes. Through FK and WS tasks with diverse input manipulations, the study shows that external inputs systematically modulate belief formation, BD predicts action, and that causal interventions can steer decisions, alongside preliminary evidence for neurofeedback-based meta-cognition. These findings provide empirical support for belief-guided agency and meta-cognition in LLMs and establish methodological groundwork for studying agency, beliefs, and meta-cognition in artificial systems, while acknowledging limitations and ethical considerations.

Abstract

Rapid advancements in large language models (LLMs) have sparked the question whether these models possess some form of consciousness. To tackle this challenge, Butlin et al. (2023) introduced a list of indicators for consciousness in artificial systems based on neuroscientific theories. In this work, we evaluate a key indicator from this list, called HOT-3, which tests for agency guided by a general belief-formation and action selection system that updates beliefs based on meta-cognitive monitoring. We view beliefs as representations in the model's latent space that emerge in response to a given input, and introduce a metric to quantify their dominance during generation. Analyzing the dynamics between competing beliefs across models and tasks reveals three key findings: (1) external manipulations systematically modulate internal belief formation, (2) belief formation causally drives the model's action selection, and (3) models can monitor and report their own belief states. Together, these results provide empirical support for the existence of belief-guided agency and meta-cognitive monitoring in LLMs. More broadly, our work lays methodological groundwork for investigating the emergence of agency, beliefs, and meta-cognition in LLMs.

Figures (8)

• Figure 1: Interpreting and testing the HOT-3 indicator in LLMs. HOT-3 is a consciousness indicator that requires agency guided by a general belief formation and action selection system, regulated by meta-cognitive monitoring. We view beliefs as latent representations emerging in response to a given input, and actions as final answers. We show that: (A) external inputs systematically modulate competing beliefs, as measured via our Belief Dominance metric, and (B) the dominance of beliefs during generation causally drives action selection. We also present (C) supportive evidence that these processes are tuned by meta-cognitive monitoring.
• Figure 2: BDDiff scores of Llama and Gemma across manipulations and tasks, split by the model's action ($a_{\text{base}}$ or $a_{\text{counter}}$). Plots are omitted in cases with $< 10$ instances or when the manipulation is not applied in the task. Differences in scores of the same manipulations between the two answer categories are statistically significant, see §\ref{['app:stat_hop2']} for details.
• Figure 3: Neurofeedback intervention results of Gemma on both tasks, showing the shifts in the predicted labels for BD($b_{\text{counter}}$) and BD($b_{\text{base}}$) with and without injecting $b_{\text{counter}}$. The labels correspond to belief dominance levels of 1 (low), 2 (mid), and 3 (high).
• Figure 4: BDDiff scores across manipulations and tasks, split by the model's action ($a_{\text{base}}$ or $a_{\text{counter}}$). (a) Scores over generation spans excluding the final token. (b) Scores over all layers. (c) Scores over all positions (including those not active). Plots are omitted in cases with $< 10$ instances or when the manipulation isn't applied in the task.
• Figure 5: Absolute BD scores during generation across manipulations, colored by the belief ($b_{\text{base}}$ or $b_{\text{counter}}$). (a) Scores for the FK task. (b) Scores for the WS task.