Mastering truss structure optimization with tree search

TL;DR

The paper addresses discrete truss topology optimization under progressive construction, formulating the objective $\|\mathbf{U}(\Omega)\|_\infty$ and leveraging a physics-based FE model. It couples generative grammar-based design synthesis with Monte Carlo Tree Search, using FE simulations as the environment and a modified UCT to efficiently navigate large discrete state spaces without an explicit transition model. Key contributions include demonstrating substantial reductions in finite element evaluations compared to Q-learning methods and achieving near-optimal performance across multiple case studies, including progressive-construction scenarios. The work suggests that MCTS with grammar guidance is a scalable tool for complex engineering design problems and has potential applications beyond planar truss lattices.

Abstract

This study investigates the combined use of generative grammar rules and Monte Carlo Tree Search (MCTS) for optimizing truss structures. Our approach accommodates intermediate construction stages characteristic of progressive construction settings. We demonstrate the significant robustness and computational efficiency of our approach compared to alternative reinforcement learning frameworks from previous research activities, such as Q-learning or deep Q-learning. These advantages stem from the ability of MCTS to strategically navigate large state spaces, leveraging the upper confidence bound for trees formula to effectively balance exploitation-exploration trade-offs. We also emphasize the importance of early decision nodes in the search tree, reflecting design choices crucial for highly performative solutions. Additionally, we show how MCTS dynamically adapts to complex and extensive state spaces without significantly affecting solution quality. While the focus of this paper is on truss optimization, our findings suggest MCTS as a powerful tool for addressing other increasingly complex engineering applications.

Figures (14)

• Figure 1: Schematic agent-environment interaction.
• Figure 2: Exemplary actions following operators $\mathcal{D}$ and $\mathcal{T}$. The current configuration $s_t$ (top) is modified either through action $a_1$ (bottom left) following the $\mathcal{D}$ operator or through action $a_2$ (bottom right) following the $\mathcal{T}$ operator, resulting in a new configuration $s_{t+1}$. In both cases, the selected truss element is $e_1$, and the chosen inactive node is $n_1$.
• Figure 3: Exemplary use of grammar rules for optimal truss design, formalized as a Markov decision process and solved through Monte Carlo tree search. The search tree construction and the corresponding truss design synthesis are achieved by repeating the four steps of selection, expansion, simulation, and backpropagation.
• Figure 4: Truss optimization - Case studies adapted from Ororbia2023: summary of design domain, seed configuration $s_0$, and target optimal design $s_T$ identified through a brute-force exhaustive search.
• Figure 5: Truss optimization - Case studies 1-6: evolution of the design objective during training, shown as the average value (solid blue line) with its one-standard-deviation credibility interval (shaded blue area), and target global minimum (dashed red line). Results averaged over 10 training runs.