Agentic Design of Compositional Machines

TL;DR

Agentic Design of Compositional Machines investigates whether large language models can learn to design functional machines by composing standardized parts within a physics-based simulator. The paper formalizes compositional machine design, introduces BesiegeField for interactive, reward-driven evaluation, and benchmarks multiple LLMs to reveal current limitations in spatial-physical reasoning. It demonstrates that RL finetuning with verifiable rewards improves design validity and performance, while highlighting persistent challenges in 3D understanding and blueprint-to-implementation translation. Collectively, the work delineates concrete directions for advancing LLM-based design agents, including richer representations, iterative reasoning routes, and data-efficient reinforcement learning in grounded design domains.

Abstract

The design of complex machines stands as both a marker of human intelligence and a foundation of engineering practice. Given recent advances in large language models (LLMs), we ask whether they, too, can learn to create. We approach this question through the lens of compositional machine design: a task in which machines are assembled from standardized components to meet functional demands like locomotion or manipulation in a simulated physical environment. With this simplification, machine design is expressed as writing XML-like code that explicitly specifies pairwise part connections. To support this investigation, we introduce BesiegeField, a testbed built on the machine-building game Besiege, which enables part-based construction, physical simulation and reward-driven evaluation. Using BesiegeField, we benchmark state-of-the-art LLMs with agentic workflows and identify key capabilities required for success, including spatial reasoning, strategic assembly, and instruction-following. As current open-source models fall short, we explore reinforcement learning (RL) as a path to improvement: we curate a cold-start dataset, conduct RL finetuning experiments, and highlight open challenges at the intersection of language, machine design, and physical reasoning.

Figures (30)

• Figure 1: The task of compositional machine design is illustrated in our BesiegeField environment. The figure shows a high-level sketch of the agentic workflow (w/ Gemini Pro 2.5), along with the resulting machines and their simulated performance. The design objective is to create a machine that throws boulders long distances.
• Figure 2: Demonstration of the machine design tasks in our experiments. (Left: car; Right: catapult).
• Figure 3: Demonstration of the default position-based representation and our construction tree representation. Parent block info is in blue and child info is in red.
• Figure 4: Example CoT of inspector agents (w/ Gemini 2.5 Pro). Blue text highlights the moderate capability of LLMs in spatial reasoning and imagined physical simulation.
• Figure 5: Example CoT of inspector agents (w/ OpenAI o3). Red text highlights reasoning errors.