A Methodology for Effective Surrogate Learning in Complex Optimization

TL;DR

This work introduces the PTME methodology to evaluate deep-learning surrogates for expensive optimization by jointly considering numerical precision and physical resource costs (Time, Memory, Energy). It applies PTME to traffic-light optimization across Málaga, Stockholm, and Paris, comparing URS and LHS sampling and using a DNN surrogate trained with up to 1M samples; key metrics include MAPE, RMSE, Kendall's $\tau$, and resource usage during training and inference. The study finds that inference costs are stable and low, training cost scales sublinearly with dataset size, and larger datasets improve precision and rank correlation, with sampling strategy differences fading at scale. Additionally, surrogate-assisted PSO (SAPSO) demonstrates energy-efficient guidance of the search with performance improving as data grow, though it may not always match full-evaluation baselines. Overall, PTME provides a practical, scalable baseline for evaluating and deploying surrogates in real-world, complex optimization problems and points toward adaptive and multi-fidelity extensions for greener AI.

Abstract

Solving complex problems requires continuous effort in developing theory and practice to cope with larger, more difficult scenarios. Working with surrogates is normal for creating a proxy that realistically models the problem into the computer. Thus, the question of how to best define and characterize such a surrogate model is of the utmost importance. In this paper, we introduce the PTME methodology to study deep learning surrogates by analyzing their Precision, Time, Memory, and Energy consumption. We argue that only a combination of numerical and physical performance can lead to a surrogate that is both a trusted scientific substitute for the real problem and an efficient experimental artifact for scalable studies. Here, we propose different surrogates for a real problem in optimally organizing the network of traffic lights in European cities and perform a PTME study on the surrogates' sampling methods, dataset sizes, and resource consumption. We further use the built surrogates in new optimization metaheuristics for decision-making in real cities. We offer better techniques and conclude that the PTME methodology can be used as a guideline for other applications and solvers.

Figures (11)

• Figure 1: Objective value distribution of 1M dataset
• Figure 2: Energy (CPU+DRAM)
• Figure 3: Time
• Figure 4: Memory
• Figure 6: MAPE