MOOT: a Repository of Many Multi-Objective Optimization Tasks
Tim Menzies, Tao Chen, Yulong Ye, Kishan Kumar Ganguly, Amirali Rayegan, Srinath Srinivasan, Andre Lustosa
TL;DR
MOOT tackles the lack of large, realistic multi-objective optimization benchmarks in software engineering by aggregating 120+ datasets from prominent venues into a single repository. The approach provides structured tasks with multiple inputs and goals, enabling scalable, data-driven exploration of trade-offs such as performance vs. cost, and to facilitate replication and cross-domain evaluation of optimization methods. The key contributions are the MOOT repository itself, data-format guidelines, baseline demonstrations illustrating label-efficient optimization, and a comprehensive roadmap of research questions spanning core optimization, human factors, industrial deployment, LLMs, and generalizability. By offering realistic, diverse SE optimization tasks and fostering community engagement, MOOT aims to accelerate robust evaluation of MSR optimization techniques and bridge research with practical deployment.
Abstract
Software engineers must make decisions that trade off competing goals (faster vs. cheaper, secure vs. usable, accurate vs. interpretable, etc.). Despite MSR's proven techniques for exploring such goals, researchers still struggle with these trade-offs. Similarly, industrial practitioners deliver sub-optimal products since they lack the tools needed to explore these trade-offs. To enable more research in this important area, we introduce MOOT, a repository of multi-objective optimization tasks taken from recent SE research papers. MOOT's tasks cover software configuration, cloud tuning, project health, process modeling, hyperparameter optimization, and more. Located at github.com/timm/moot, MOOT's current 120+ tasks are freely available under an MIT license (and we invite community contributions). As shown here, this data enables dozens of novel research questions.