Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning

Aditya Sharma, Ananya Gupta, Chengyu Wang, Chiamaka Adebayo, Jakub Kowalski

TL;DR

This work addresses the limitation that LLMs lack grounded physical understanding by introducing Causal World Model Induction (CWMI), which embeds a Causal Physics Module (CPM) inside a frozen LLM and trains it with a Causal Intervention Loss. The CPM acts as a latent-space physics engine, learning causal dynamics through state-prediction and counterfactual reasoning, enabling zero-shot physical QA that leverages internal simulation. On PIQA and the newly proposed PhysiCa-Bench, CWMI substantially outperforms baselines, with high CCS and low FSPA, demonstrating that explicit causal world modeling yields more robust, generalizable physical reasoning. The work highlights the value of architectural modularity (LLM as language interface, CPM as physics engine) and skillful use of multimodal data, offering practical gains for safe, reliable embodied AI systems.

Abstract

Large Language Models (LLMs), despite their advanced linguistic capabilities, fundamentally lack an intuitive understanding of physical dynamics, which limits their effectiveness in real-world scenarios that require causal reasoning. In this paper, we introduce Causal World Model Induction (CWMI), a novel framework designed to embed an explicit model of causal physics within an LLM. Our approach incorporates a dedicated Causal Physics Module (CPM) and a new training objective called Causal Intervention Loss, encouraging the model to learn cause-and-effect relationships from multimodal data. By training the model to predict the outcomes of hypothetical interventions instead of merely capturing statistical correlations, CWMI develops a robust internal representation of physical laws. Experimental results show that CWMI significantly outperforms state-of-the-art LLMs on zero-shot physical reasoning tasks, including the PIQA benchmark and our newly proposed PhysiCa-Bench dataset. These findings demonstrate that inducing a causal world model is a critical step toward more reliable and generalizable AI systems.

Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning

TL;DR

This work addresses the limitation that LLMs lack grounded physical understanding by introducing Causal World Model Induction (CWMI), which embeds a Causal Physics Module (CPM) inside a frozen LLM and trains it with a Causal Intervention Loss. The CPM acts as a latent-space physics engine, learning causal dynamics through state-prediction and counterfactual reasoning, enabling zero-shot physical QA that leverages internal simulation. On PIQA and the newly proposed PhysiCa-Bench, CWMI substantially outperforms baselines, with high CCS and low FSPA, demonstrating that explicit causal world modeling yields more robust, generalizable physical reasoning. The work highlights the value of architectural modularity (LLM as language interface, CPM as physics engine) and skillful use of multimodal data, offering practical gains for safe, reliable embodied AI systems.

Abstract

Large Language Models (LLMs), despite their advanced linguistic capabilities, fundamentally lack an intuitive understanding of physical dynamics, which limits their effectiveness in real-world scenarios that require causal reasoning. In this paper, we introduce Causal World Model Induction (CWMI), a novel framework designed to embed an explicit model of causal physics within an LLM. Our approach incorporates a dedicated Causal Physics Module (CPM) and a new training objective called Causal Intervention Loss, encouraging the model to learn cause-and-effect relationships from multimodal data. By training the model to predict the outcomes of hypothetical interventions instead of merely capturing statistical correlations, CWMI develops a robust internal representation of physical laws. Experimental results show that CWMI significantly outperforms state-of-the-art LLMs on zero-shot physical reasoning tasks, including the PIQA benchmark and our newly proposed PhysiCa-Bench dataset. These findings demonstrate that inducing a causal world model is a critical step toward more reliable and generalizable AI systems.

Paper Structure

This paper contains 31 sections, 4 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reasoning in Large Language Models
  4. World Models and Physical Prediction
  5. Causality and Multimodal Grounding
  6. Experimental Setup
  7. Datasets
  8. Training and Fine-tuning
  9. Zero-Shot Evaluation
  10. Implementation Details and Hardware
  11. Evaluation Metrics
  12. Methodology
  13. System Architecture
  14. The Causal Physics Module (CPM)
  15. State Representation
  16. ...and 16 more sections

Figures (4)

  • Figure 1: Rich, icon‑enhanced flowchart of the CWMI framework. The diagram illustrates the sequence of operations—from “Input Text” through “LLM Encoding,” “Projection Layer,” “Causal Physics Module,” to final “Output”—with color‑coded modules and visual cues to emphasize each stage’s function.
  • Figure 2: The overall architecture of the Causal World Model Induction (CWMI) framework. An input text describing a physical scene is processed by the frozen LLM backbone. The LLM's final hidden state, which encodes a rich semantic representation of the scene, is used to initialize the state of the trainable Causal Physics Module (CPM). The CPM, acting as a latent physics simulator, predicts the final state of the system. The entire model is trained with a composite loss function, including a key Causal Intervention Loss derived from counterfactual pairs.
  • Figure 3: Zero-Shot Reasoning Accuracy on PIQA.
  • Figure 4: CWMI Performance vs CPM Capacity: Causal Consistency Score