EMIT: Micro-Invasive Database Configuration Tuning
Jian Geng, Hongzhi Wang, Yu Yan
TL;DR
EMIT tackles the problem of database knob tuning being NP-hard and disruptively intrusive by introducing a micro-invasive approach that re-runs tuning on a Cloned DB using synthesized workloads, thereby avoiding direct production workload exposure. The method combines workload synthesis, Experience Enhanced tuning with common knowledge extraction, and a recursive configuration replacement strategy, all guided by a modified SMAC Bayesian optimization that leverages historical knowledge. Key contributions include (i) a workload synthesis technique to emulate target workloads without data leakage, (ii) a recursive configuration selection mechanism to efficiently locate good configurations, (iii) a common-knowledge transfer framework to prune the search space, and (iv) an Experience Enhanced SMAC that decays reliance on history to speed convergence. Experiments on YCSB, TPCC, and Twitter demonstrate faster convergence and fewer production updates while maintaining or improving performance, highlighting EMIT’s practical potential for safe, scalable database tuning in production environments.
Abstract
The process of database knob tuning has always been a challenging task. Recently, database knob tuning methods has emerged as a promising solution to mitigate these issues. However, these methods still face certain limitations.On one hand, when applying knob tuning algorithms to optimize databases in practice, it either requires frequent updates to the database or necessitates acquiring database workload and optimizing through workload replay. The former approach involves constant exploration and updating of database configurations, inevitably leading to a decline in database performance during optimization. The latter, on the other hand, requires the acquisition of workload data, which could lead to data leakage issues. Moreover, the hyperparameter configuration space for database knobs is vast, making it challenging for optimizers to converge. These factors significantly hinder the practical implementation of database tuning. To address these concerns, we proposes an efficient and micro-invasive knob tuning method. This method relies on workload synthesis on cloned databases to simulate the workload that needs tuning, thus minimizing the intrusion on the database. And we utilizing a configuration replacement strategy to filter configuration candidates that perform well under the synthesized workload to find best configuration. And during the tuning process, we employ a knowledge transfer method to extract a common high-performance space, to boost the convergence of the optimizer.