Structure-Aware Cooperative Ensemble Evolutionary Optimization on Combinatorial Problems with Multimodal Large Language Models

TL;DR

This paper tackles the challenge of structure-aware optimization on graph-structured combinatorial problems by adopting an image-based encoding with multimodal LLMs as evolutionary operators. It introduces a cooperative master-worker framework over multiple sparsified graph views and an ensemble of network layouts to transfer knowledge across domains and reduce layout bias. The approach is validated on influence maximization and auxiliary toy tasks, showing improved solution quality, robustness, and generalizability across different problem types. The results suggest that integrating MLLMs with multi-view sparsification and layout ensembles can significantly enhance discrete graph optimization in practice.

Abstract

Evolutionary algorithms (EAs) have proven effective in exploring the vast solution spaces typical of graph-structured combinatorial problems. However, traditional encoding schemes, such as binary or numerical representations, often fail to straightforwardly capture the intricate structural properties of networks. Through employing the image-based encoding to preserve topological context, this study utilizes multimodal large language models (MLLMs) as evolutionary operators to facilitate structure-aware optimization over graph data. To address the visual clutter inherent in large-scale network visualizations, we leverage graph sparsification techniques to simplify structures while maintaining essential structural features. To further improve robustness and mitigate bias from different sparsification views, we propose a cooperative evolutionary optimization framework that facilitates cross-domain knowledge transfer and unifies multiple sparsified variants of diverse structures. Additionally, recognizing the sensitivity of MLLMs to network layout, we introduce an ensemble strategy that aggregates outputs from various layout configurations through consensus voting. Finally, experiments on real-world networks through various tasks demonstrate that our approach improves both the quality and reliability of solutions in MLLM-driven evolutionary optimization.

Figures (14)

• Figure 1: Diagram of the cooperative evolutionary optimization. The master unit stores the elite solutions and is responsible for knowledge transfer across subprocessors.
• Figure 2: Diagram of the ensemble strategy. Each layout is fed into MLLMs separately, and the final outcome is determined using the consensus voting strategy.
• Figure 3: Diagram of the MLLM-based evolutionary optimization on influence maximization. The solution and its underlying network are jointly represented in the image format.
• Figure 4: Comparison of cooperative and vanilla evolutionary optimization (EO). Shaded regions represent the Standard Error of the Mean (SEM). The number of solutions in each Co-EO subprocessor population is set to half that of the single-domain configuration, since two sparsified networks are employed.
• Figure 5: Comparison of MLLM-based optimization performance across different graph layouts. Shaded regions represent the Standard Error of the Mean (SEM).

Theorems & Definitions (1)

• Remark 1