Can Large Language Models Solve Engineering Equations? A Systematic Comparison of Direct Prediction and Solver-Assisted Approaches

TL;DR

This study systematically compares direct end-to-end LLM numerical prediction against solver-assisted architectures for solving transcendental equations encountered in engineering. Across six state-of-the-art models and 100 problems in seven domains, direct predictions yield high mean relative errors, while a hybrid approach—where LLMs formulate equations and provide initial guesses and Newton-Raphson iterations perform the computation—achieves substantial error reductions ($ ext{MRE}$ from about $0.23$–$0.30$ vs $0.76$–$1.24$). The improvements are domain-dependent, with electronics showing the largest gains due to exponential sensitivity and some domains like fluid mechanics benefiting from pattern-driven approximations. The results support deploying LLMs as intelligent interfaces to classical solvers rather than as standalone computational engines, enabling accessible, reliable numerical engineering workflows while preserving mathematical guarantees. The work highlights opportunities for iterative LLM-solver dialogue, broader equation classes, and continued assessment as model capabilities evolve.

Abstract

Transcendental equations requiring iterative numerical solution pervade engineering practice, from fluid mechanics friction factor calculations to orbital position determination. We systematically evaluate whether Large Language Models can solve these equations through direct numerical prediction or whether a hybrid architecture combining LLM symbolic manipulation with classical iterative solvers proves more effective. Testing six state-of-the-art models (GPT-5.1, GPT-5.2, Gemini-3-Flash, Gemini-2.5-Lite, Claude-Sonnet-4.5, Claude-Opus-4.5) on 100 problems spanning seven engineering domains, we compare direct prediction against solver-assisted computation where LLMs formulate governing equations and provide initial conditions while Newton-Raphson iteration performs numerical solution. Direct prediction yields mean relative errors of 0.765 to 1.262 across models, while solver-assisted computation achieves 0.225 to 0.301, representing error reductions of 67.9% to 81.8%. Domain-specific analysis reveals dramatic improvements in Electronics (93.1%) due to exponential equation sensitivity, contrasted with modest gains in Fluid Mechanics (7.2%) where LLMs exhibit effective pattern recognition. These findings establish that contemporary LLMs excel at symbolic manipulation and domain knowledge retrieval but struggle with precision-critical iterative arithmetic, suggesting their optimal deployment as intelligent interfaces to classical numerical solvers rather than standalone computational engines.

Figures (4)

• Figure 1: Computational paradigms for LLM-based equation solving. (a) Direct approach: LLM generates numerical answer directly. (b) Solver-assisted approach: LLM formulates symbolic equation and provides initial guess, Newton-Raphson performs iterative solution.
• Figure 2: Performance Comparison: Direct LLM vs. Solver-Assisted Computation. Error reduction percentages show consistent dramatic improvement from hybrid architecture across all six models.
• Figure 3: Domain-Specific Performance (Log Scale). Averaged across models, showing substantial variation in hybrid architecture effectiveness. Logarithmic scale accommodates extreme Electronics errors while maintaining clarity for other domains.
• Figure 4: Query-Level Performance: (a) Direct LLM prediction exhibits extreme variability with frequent error spikes exceeding 5.0 (capped for visualization). (b) Solver-Assisted approach shows dramatically reduced error variability and magnitude across all problems.