Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AutoEP: LLMs-Driven Automation of Hyperparameter Evolution for Metaheuristic Algorithms

Zhenxing Xu, Yizhe Zhang, Weidong Bao, Hao Wang, Ming Chen, Haoran Ye, Wenzheng Jiang, Hui Yan, Ji Wang

Abstract

Dynamically configuring algorithm hyperparameters is a fundamental challenge in computational intelligence. While learning-based methods offer automation, they suffer from prohibitive sample complexity and poor generalization. We introduce AutoEP, a novel framework that bypasses training entirely by leveraging Large Language Models (LLMs) as zero-shot reasoning engines for algorithm control. AutoEP's core innovation lies in a tight synergy between two components: (1) an online Exploratory Landscape Analysis (ELA) module that provides real-time, quantitative feedback on the search dynamics, and (2) a multi-LLM reasoning chain that interprets this feedback to generate adaptive hyperparameter strategies. This approach grounds high-level reasoning in empirical data, mitigating hallucination. Evaluated on three distinct metaheuristics across diverse combinatorial optimization benchmarks, AutoEP consistently outperforms state-of-the-art tuners, including neural evolution and other LLM-based methods. Notably, our framework enables open-source models like Qwen3-30B to match the performance of GPT-4, demonstrating a powerful and accessible new paradigm for automated hyperparameter design. Our code is available at https://github.com/YiZheZhang12/AutoEP.

AutoEP: LLMs-Driven Automation of Hyperparameter Evolution for Metaheuristic Algorithms

Abstract

Dynamically configuring algorithm hyperparameters is a fundamental challenge in computational intelligence. While learning-based methods offer automation, they suffer from prohibitive sample complexity and poor generalization. We introduce AutoEP, a novel framework that bypasses training entirely by leveraging Large Language Models (LLMs) as zero-shot reasoning engines for algorithm control. AutoEP's core innovation lies in a tight synergy between two components: (1) an online Exploratory Landscape Analysis (ELA) module that provides real-time, quantitative feedback on the search dynamics, and (2) a multi-LLM reasoning chain that interprets this feedback to generate adaptive hyperparameter strategies. This approach grounds high-level reasoning in empirical data, mitigating hallucination. Evaluated on three distinct metaheuristics across diverse combinatorial optimization benchmarks, AutoEP consistently outperforms state-of-the-art tuners, including neural evolution and other LLM-based methods. Notably, our framework enables open-source models like Qwen3-30B to match the performance of GPT-4, demonstrating a powerful and accessible new paradigm for automated hyperparameter design. Our code is available at https://github.com/YiZheZhang12/AutoEP.

Paper Structure

This paper contains 29 sections, 5 equations, 15 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. AutoEP
  4. Grounding Reasoning with Quantitative Search Dynamics
  5. Fitness Distribution Features.
  6. Fitness Landscape and Diversity Features
  7. Search Progress Feature
  8. A Closed-Loop Architecture for LLM-Driven Control
  9. Decomposing Control Logic with a Chain of Reasoning
  10. Experiment
  11. Experimental settings
  12. Results
  13. Validation on TSP
  14. Validation on CVRP, FSSP, and UAV trajectory optimization
  15. Ablation Studies: Deconstructing AutoEP's Performance
  16. ...and 14 more sections

Figures (15)

  • Figure 1: Process of hyperparameter tuning in metaheuristic algorithms using AutoEP.
  • Figure 2: The AutoEP Framework.
  • Figure 3: Demonstration of CoR.
  • Figure 4: Comparison of Experimental Results Across Different LLMs. The baseline algorithm for adjustment is GA-2opt.
  • Figure 5: Comparison of hyperparameter tuning frequencies(UAV-300).
  • ...and 10 more figures