Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FactorSmith: Agentic Simulation Generation via Markov Decision Process Decomposition with Planner-Designer-Critic Refinement

Ali Shamsaddinlou, Morteza NourelahiAlamdari

Abstract

Generating executable simulations from natural language specifications remains a challenging problem due to the limited reasoning capacity of large language models (LLMs) when confronted with large, interconnected codebases. This paper presents FactorSmith, a framework that synthesizes playable game simulations in code from textual descriptions by combining two complementary ideas: factored POMDP decomposition for principled context reduction and a hierarchical planner-designer-critic agentic workflow for iterative quality refinement at every generation step. Drawing on the factored partially observable Markov decision process (POMDP) representation introduced by FactorSim [Sun et al., 2024], the proposed method decomposes a simulation specification into modular steps where each step operates only on a minimal subset of relevant state variables, limiting the context window that any single LLM call must process. Inspired by the agentic trio architecture of SceneSmith [Pfaff et al., 2025], FactorSmith embeds within every factored step a three-agent interaction: a planner that orchestrates workflow, a designer that proposes code artifacts, and a critic that evaluates quality through structured scoring, enabling iterative refinement with checkpoint rollback. This paper formalizes the combined approach, presents the mathematical framework underpinning context selection and agentic refinement, and describes the open-source implementation. Experiments on the PyGame Learning Environment benchmark demonstrate that FactorSmith generates simulations with improved prompt alignment, fewer runtime errors, and higher code quality compared to non-agentic factored baselines.

FactorSmith: Agentic Simulation Generation via Markov Decision Process Decomposition with Planner-Designer-Critic Refinement

Abstract

Generating executable simulations from natural language specifications remains a challenging problem due to the limited reasoning capacity of large language models (LLMs) when confronted with large, interconnected codebases. This paper presents FactorSmith, a framework that synthesizes playable game simulations in code from textual descriptions by combining two complementary ideas: factored POMDP decomposition for principled context reduction and a hierarchical planner-designer-critic agentic workflow for iterative quality refinement at every generation step. Drawing on the factored partially observable Markov decision process (POMDP) representation introduced by FactorSim [Sun et al., 2024], the proposed method decomposes a simulation specification into modular steps where each step operates only on a minimal subset of relevant state variables, limiting the context window that any single LLM call must process. Inspired by the agentic trio architecture of SceneSmith [Pfaff et al., 2025], FactorSmith embeds within every factored step a three-agent interaction: a planner that orchestrates workflow, a designer that proposes code artifacts, and a critic that evaluates quality through structured scoring, enabling iterative refinement with checkpoint rollback. This paper formalizes the combined approach, presents the mathematical framework underpinning context selection and agentic refinement, and describes the open-source implementation. Experiments on the PyGame Learning Environment benchmark demonstrate that FactorSmith generates simulations with improved prompt alignment, fewer runtime errors, and higher code quality compared to non-agentic factored baselines.
Paper Structure (42 sections, 1 theorem, 11 equations, 1 figure, 3 tables, 2 algorithms)

This paper contains 42 sections, 1 theorem, 11 equations, 1 figure, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-Based Simulation Generation.
  4. Agentic Code Generation.
  5. Agentic Scene and Environment Generation.
  6. Structured LLM Reasoning.
  7. Background
  8. POMDP Formulation for Simulations
  9. The Planner--Designer--Critic Pattern
  10. Method: FactorSmith
  11. Pipeline Overview
  12. Phase 1: High-Level Decomposition.
  13. Phase 2: Factored Step Execution.
  14. Phase 3: Assembly and Validation.
  15. Agentic Refinement Within Each Factored Step
  16. ...and 27 more sections

Key Result

Proposition 1

Under the assumption that the critic's scoring function is calibrated (i.e., higher scores correspond to higher actual quality) and the designer's revision monotonically improves the aspect identified by the critic, the checkpoint rollback mechanism in Algorithm alg:agentic-step ensures that the tot

Figures (1)

  • Figure 1: Architecture of FactorSmith. A game prompt is first decomposed into modular steps via Chain-of-Thought. Each step is processed by a State Agent (planner/designer/critic trio) that identifies relevant and new state variables, selecting only the contextual subset from the session database. The Decompose Agent then splits the step into three MVC components---input logic, state transition, and UI rendering---each handled by its own agentic trio (Input Logic Agent, Logic Agent, UI Agent). All generated functions and state variables are persisted in the session database, enabling factored context selection for subsequent steps.

Theorems & Definitions (3)

  • Definition 1: Factored POMDP sun2024factorsim
  • Proposition 1: Quality Monotonicity
  • proof