BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction

TL;DR

BuildArena addresses the gap in evaluating LLMs for language-driven construction under physics constraints by introducing a physics-aligned interactive benchmark. It combines a Besiege-based simulation with a 3D Spatial Geometric Computation Library and an agentic workflow to enable end-to-end language-to-assembly testing across static and dynamic tasks. The framework offers a customizable task design, hierarchical difficulty, and logging-based evaluation, evaluated across eight frontier LLMs to reveal both potential and current limitations in physical construction reasoning. The results highlight meaningful progress in language-driven 3D construction while underscoring persistent challenges in precision, hierarchical assembly, and efficient cost-utility trade-offs, signaling a path toward more capable, physics-aware AI systems in engineering contexts.

Abstract

Engineering construction automation aims to transform natural language specifications into physically viable structures, requiring complex integrated reasoning under strict physical constraints. While modern LLMs possess broad knowledge and strong reasoning capabilities that make them promising candidates for this domain, their construction competencies remain largely unevaluated. To address this gap, we introduce BuildArena, the first physics-aligned interactive benchmark designed for language-driven engineering construction. It contributes to the community in four aspects: (1) a highly customizable benchmarking framework for in-depth comparison and analysis of LLMs; (2) an extendable task design strategy spanning static and dynamic mechanics across multiple difficulty tiers; (3) a 3D Spatial Geometric Computation Library for supporting construction based on language instructions; (4) a baseline LLM agentic workflow that effectively evaluates diverse model capabilities. On eight frontier LLMs, BuildArena comprehensively evaluates their capabilities for language-driven and physics-grounded construction automation. The project page is at https://build-arena.github.io/.

Figures (10)

• Figure 1: Examples of BuildArena's construction results by LLMs, covering three tasks: Lift (left subfigure), Transport (upper right), and Support (lower right).
• Figure 2: Illustration of our BuildArena framework. It contains three parts: (1) Task definition; (2) LLM-based Construction; (3) Simulation-based Evaluation. The arrows represent our pipeline. Components in dashed boxes, i.e., task type, LLM agentic workflow, and simulator, could be customized by users. Details of the construction procedure is shown in Figure \ref{['fig:construction_procedure']}.
• Figure 3: Difficulty profiles of the three task Transport, Support, and Lift across six engineering dimensions: Quantification, Redundancy, Scale, Modularity, Precision, and Ambiguity. Each radar chart illustrates how difficulty escalates from Lv.1 (blue) to Lv.2 (purple) and Lv.3 (red).
• Figure 4: Details of the construction procedure in Figure \ref{['fig:pipeline']}. Our designed workflow (bottom row) contains five collaborative LLM entities, serves as a baseline for future user-customized alternatives. The text-based construction space (top row) has two transferable formats: code for physics-aligned spatial geometric computation, and natural language for LLM interface compatibility.
• Figure 5: Example of the construction process. The rocket is constructed by Grok-4 for the Lift task under the Lv.2 (Medium) difficulty level. More examples are presented in Figure \ref{['fig:result_construction_procedures']}.