Large Language Models as Particle Swarm Optimizers
Yamato Shinohara, Jinglue Xu, Tianshui Li, Hitoshi Iba
TL;DR
LMPSO embeds a large language model into Particle Swarm Optimization by updating particle positions through velocity-promoting prompts, enabling a prompt-driven, hyper-heuristic search across diverse problem classes. The method uses a meta-prompt to steer the LLM and preserves the PSO structure, achieving competitive results on small combinatorial TSP problems and delivering strong improvements in heuristic refinement and symbolic regression. Across experiments, LMPSO demonstrates that LLMs can generate effective, concise, and sometimes novel solution strategies when integrated with swarm coordination, expanding PSO's applicability to structured representations and natural-language search spaces. The work highlights promising directions for broadening problem domains, assessing LLM capabilities, and applying advanced prompting to enhance optimization in practical contexts.
Abstract
Optimization problems often require domain-specific expertise to design problem-dependent methodologies. Recently, several approaches have gained attention by integrating large language models (LLMs) into genetic algorithms. Building on this trend, we introduce Language Model Particle Swarm Optimization (LMPSO), a novel method that incorporates an LLM into the swarm intelligence framework of Particle Swarm Optimization (PSO). In LMPSO, the velocity of each particle is represented as a prompt that generates the next candidate solution, leveraging the capabilities of an LLM to produce solutions in accordance with the PSO paradigm. This integration enables an LLM-driven search process that adheres to the foundational principles of PSO. The proposed LMPSO approach is evaluated across multiple problem domains, including the Traveling Salesman Problem (TSP), heuristic improvement for TSP, and symbolic regression. These problems are traditionally challenging for standard PSO due to the structured nature of their solutions. Experimental results demonstrate that LMPSO is particularly effective for solving problems where solutions are represented as structured sequences, such as mathematical expressions or programmatic constructs. By incorporating LLMs into the PSO framework, LMPSO establishes a new direction in swarm intelligence research. This method not only broadens the applicability of PSO to previously intractable problems but also showcases the potential of LLMs in addressing complex optimization challenges.