DUCTILE: Agentic LLM Orchestration of Engineering Analysis in Product Development Practice

TL;DR

A DUCTILE (Delegated, User-supervised Coordination of Tool- and document-Integrated LLM-Enabled) agentic orchestration, an approach for developing, executing and evaluating LLM-based agentic automation support of engineering analysis tasks.

Abstract

Engineering analysis automation in product development relies on rigid interfaces between tools, data formats and documented processes. When these interfaces change, as they routinely do as the product evolves in the engineering ecosystem, the automation support breaks. This paper presents a DUCTILE (Delegated, User-supervised Coordination of Tool- and document-Integrated LLM-Enabled) agentic orchestration, an approach for developing, executing and evaluating LLM-based agentic automation support of engineering analysis tasks. The approach separates adaptive orchestration, performed by the LLM agent, from deterministic execution, performed by verified engineering tools. The agent interprets documented design practices, inspects input data and adapts the processing path, while the engineer supervises and exercises final judgment. DUCTILE is demonstrated on an industrial structural analysis task at an aerospace manufacturer, where the agent handled input deviations in format, units, naming conventions and methodology that would break traditional scripted pipelines. Evaluation against expert-defined acceptance criteria and deployment with practicing engineers confirm that the approach produces correct, methodologically compliant results across repeated independent runs. The paper discusses practical consequences of adopting agentic automation, including unintended effects on the nature of engineering work and the tension between removing mundane tasks and creating an exhausting supervisory role.

Figures (9)

• Figure 1: Schematic of the LLM inference loop. The model takes the full context window as input, produces a probability distribution over the next token, samples one token according to a sampling strategy, appends it to the context window, and repeats until a stop token (<EOS>) is generated. The resulting sequence of tokens forms the response.
• Figure 2: Comparison of LLM responses with and without thinking mode enabled. The thinking scratchpad (dashed box) allows the model to self-critique and reconsider the autoregressive initial response before committing to a final answer.
• Figure 3: Comparison of LLM responses with and without tool calling. Without tools, the model relies on parametric knowledge from training weights, returning a room-temperature value. With tool calling, the model queries an external materials database for temperature-dependent data, grounding its response on verified external information.
• Figure 4: Architecture of an agentic application. The inference engine processes user messages through the LLM within a context window, generating either final answers or tool calls. Tool execution is handled internally or through external interfaces that connect to external resources including file systems, knowledge bases, tool repositories and system interfaces.
• Figure 5: Top: Views of the component in the physical domain. Bottom: Activity View of the TRS Strength evaluation. Only a subset of the inputs, documents and tools are shown for visual clarity. Highlighted in orange is the load processing step where the use case focuses.