Yukthi Opus: A Multi-Chain Hybrid Metaheuristic for Large-Scale NP-Hard Optimization

TL;DR

Yukthi Opus (YO) introduces a three-layer hybrid metaheuristic for expensive, black-box NP-hard optimization by coupling MCMC-based global exploration, greedy local refinement, and adaptive simulated annealing with reheating within a structured two-phase workflow. The method uses burn-in exploration, post-burnin selection, a spatial blacklist, and multi-chain parallelism to achieve robust performance and predictable evaluation budgets across diverse problem classes, including Rastrigin, TSP, and Rosenbrock. Ablation studies demonstrate that MCMC and greedy search are essential for solution quality, while SA and multi-chain execution mainly boost stability and reduce variance; YO trades some best-case performance for improved robustness. Compared with state-of-the-art baselines, YO delivers fast runtime and competitive quality on large multimodal problems, while BayesOpt dominates smooth, low-dimensional landscapes; YO is thus well-suited for constrained-budget, gradient-free optimization where robustness and exploration are critical.

Abstract

We present Yukthi Opus (YO), a multi-chain hybrid metaheuristic designed for NP-hard optimization under explicit evaluation budget constraints. YO integrates three complementary mechanisms in a structured two-phase architecture: Markov Chain Monte Carlo (MCMC) for global exploration, greedy local search for exploitation, and simulated annealing with adaptive reheating to enable controlled escape from local minima. A dedicated burn-in phase allocates evaluations to probabilistic exploration, after which a hybrid optimization loop refines promising candidates. YO further incorporates a spatial blacklist mechanism to avoid repeated evaluation of poor regions and a multi-chain execution strategy to improve robustness and reduce sensitivity to initialization. We evaluate YO on three benchmarks: the Rastrigin function (5D) with ablation studies, the Traveling Salesman Problem with 50 to 200 cities, and the Rosenbrock function (5D) with comparisons against established optimizers including CMA-ES, Bayesian optimization, and accelerated particle swarm optimization. Results show that MCMC exploration and greedy refinement are critical for solution quality, while simulated annealing and multi-chain execution primarily improve stability and variance reduction. Overall, YO achieves competitive performance on large and multimodal problems while maintaining predictable evaluation budgets, making it suitable for expensive black-box optimization settings.

Figures (9)

• Figure 1: YO Hybrid Optimizer workflow showing two-phase architecture with adaptive mechanisms.
• Figure 2: Ablation results showing performance degradation when components removed from YO. Box plots show distribution across 30 runs per variant.
• Figure 3: TSP $N=50$, seed 42: Convergence comparison showing YO vs baselines.
• Figure 4: TSP $N=50$, seed 42: Best path found by YO Hybrid.
• Figure 5: TSP $N=100$, seed 101: Convergence comparison.