Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Particles to Agents: Hallucination as a Metric for Cognitive Friction in Spatial Simulation

Javier Argota Sánchez-Vaquerizo, Luis Borunda Monsivais

TL;DR

Traditional architectural simulations model occupants as deterministic particles; this work introduces Agentic Environmental Simulations using LLM-based agents to predict semantically grounded next states in spatial pipelines. It replaces chronological physics with Episodic Spatial Reasoning around narrative beats and cognitive boundaries, supported by a dual-process compute strategy. A central innovation is the Cognitive Friction metric, $C_f = 1 - \text{sim}(E_{gen}, R_{phys})$, which identifies Phantom Affordances via heatmaps and guides semiotic alignment. The authors advocate a Cognitive Orchestration framework that emphasizes interpretability, opt-out, demographic equity, and audit trails to preserve autonomy and cognitive integrity, with plans for pilot validation in real environments.

Abstract

Traditional architectural simulations (e.g. Computational Fluid Dynamics, evacuation, structural analysis) model elements as deterministic physics-based "particles" rather than cognitive "agents". To bridge this, we introduce \textbf{Agentic Environmental Simulations}, where Large Multimodal generative models actively predict the next state of spatial environments based on semantic expectation. Drawing on examples from accessibility-oriented AR pipelines and multimodal digital twins, we propose a shift from chronological time-steps to Episodic Spatial Reasoning, where simulations advance through meaningful, surprisal-triggered events. Within this framework we posit AI hallucinations as diagnostic tools. By formalizing the \textbf{Cognitive Friction} ($C_f$) it is possible to reveal "Phantom Affordances", i.e. semiotic ambiguities in built space. Finally, we challenge current HCI paradigms by treating environments as dynamic cognitive partners and propose a human-centered framework of cognitive orchestration for designing AI-driven simulations that preserve autonomy, affective clarity, and cognitive integrity.

From Particles to Agents: Hallucination as a Metric for Cognitive Friction in Spatial Simulation

TL;DR

Traditional architectural simulations model occupants as deterministic particles; this work introduces Agentic Environmental Simulations using LLM-based agents to predict semantically grounded next states in spatial pipelines. It replaces chronological physics with Episodic Spatial Reasoning around narrative beats and cognitive boundaries, supported by a dual-process compute strategy. A central innovation is the Cognitive Friction metric, , which identifies Phantom Affordances via heatmaps and guides semiotic alignment. The authors advocate a Cognitive Orchestration framework that emphasizes interpretability, opt-out, demographic equity, and audit trails to preserve autonomy and cognitive integrity, with plans for pilot validation in real environments.

Abstract

Traditional architectural simulations (e.g. Computational Fluid Dynamics, evacuation, structural analysis) model elements as deterministic physics-based "particles" rather than cognitive "agents". To bridge this, we introduce \textbf{Agentic Environmental Simulations}, where Large Multimodal generative models actively predict the next state of spatial environments based on semantic expectation. Drawing on examples from accessibility-oriented AR pipelines and multimodal digital twins, we propose a shift from chronological time-steps to Episodic Spatial Reasoning, where simulations advance through meaningful, surprisal-triggered events. Within this framework we posit AI hallucinations as diagnostic tools. By formalizing the \textbf{Cognitive Friction} () it is possible to reveal "Phantom Affordances", i.e. semiotic ambiguities in built space. Finally, we challenge current HCI paradigms by treating environments as dynamic cognitive partners and propose a human-centered framework of cognitive orchestration for designing AI-driven simulations that preserve autonomy, affective clarity, and cognitive integrity.
Paper Structure (7 sections, 1 equation, 1 figure)

This paper contains 7 sections, 1 equation, 1 figure.

Table of Contents

  1. Architectural simulations are becoming agentic
  2. From Chronological Physics to Episodic Cognition
  3. Not a bug, but a feature: Hallucination as Design Heuristic
  4. Quantifying Cognitive Friction: The Hallucination-Reality Gap
  5. Ethical Implications: From Optimization to Orchestration
  6. Discussion: Validation and Limitations
  7. Conclusion

Figures (1)

  • Figure 1: The Cognitive Friction Pipeline. The simulation operates on a physics autopilot (System 1) until prediction error (surprisal) exceeds a threshold $\tau$Zacks2007EventSegmentation. This triggers the VLM (System 2) to generate an expectation ($E_{gen}$). The semantic divergence ($\Delta = 1 - sim$) from ground truth ($R_{phys}$) quantifies the Cognitive Friction ($C_f$); high friction reveals a "Phantom Affordance".