Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyperZero: A Customized End-to-End Auto-Tuning System for Recommendation with Hourly Feedback

Xufeng Cai, Ziwei Guan, Lei Yuan, Ali Selman Aydin, Tengyu Xu, Boying Liu, Wenbo Ren, Renkai Xiang, Songyi He, Haichuan Yang, Serena Li, Mingze Gao, Yue Weng, Ji Liu

TL;DR

HyperZero addresses the challenge of tuning the value-model hyperparameters in large-scale recommender systems under tight deployment timelines. It introduces a data normalization strategy based on semi-i.i.d. metric deltas, and a constrained zeroth-order optimizer that combines Gaussian processes with Thompson sampling to handle non-i.i.d. hourly feedback and multiple objectives. The system supports asynchronous parallel exploration to mitigate latency and delays, enabling rapid convergence in days rather than weeks. Empirical results in both synthetic and industrial settings show superior objective gains and robust constraint satisfaction compared to baselines, highlighting practical impact for production-grade recommendation systems.

Abstract

Modern recommendation systems can be broadly divided into two key stages: the ranking stage, where the system predicts various user engagements (e.g., click-through rate, like rate, follow rate, watch time), and the value model stage, which aggregates these predictive scores through a function (e.g., a linear combination defined by a weight vector) to measure the value of each content by a single numerical score. Both stages play roughly equally important roles in real industrial systems; however, how to optimize the model weights for the second stage still lacks systematic study. This paper focuses on optimizing the second stage through auto-tuning technology. Although general auto-tuning systems and solutions - both from established production practices and open-source solutions - can address this problem, they typically require weeks or even months to identify a feasible solution. Such prolonged tuning processes are unacceptable in production environments for recommendation systems, as suboptimal value models can severely degrade user experience. An effective auto-tuning solution is required to identify a viable model within 2-3 days, rather than the extended timelines typically associated with existing approaches. In this paper, we introduce a practical auto-tuning system named HyperZero that addresses these time constraints while effectively solving the unique challenges inherent in modern recommendation systems. Moreover, this framework has the potential to be expanded to broader tuning tasks within recommendation systems.

HyperZero: A Customized End-to-End Auto-Tuning System for Recommendation with Hourly Feedback

TL;DR

HyperZero addresses the challenge of tuning the value-model hyperparameters in large-scale recommender systems under tight deployment timelines. It introduces a data normalization strategy based on semi-i.i.d. metric deltas, and a constrained zeroth-order optimizer that combines Gaussian processes with Thompson sampling to handle non-i.i.d. hourly feedback and multiple objectives. The system supports asynchronous parallel exploration to mitigate latency and delays, enabling rapid convergence in days rather than weeks. Empirical results in both synthetic and industrial settings show superior objective gains and robust constraint satisfaction compared to baselines, highlighting practical impact for production-grade recommendation systems.

Abstract

Modern recommendation systems can be broadly divided into two key stages: the ranking stage, where the system predicts various user engagements (e.g., click-through rate, like rate, follow rate, watch time), and the value model stage, which aggregates these predictive scores through a function (e.g., a linear combination defined by a weight vector) to measure the value of each content by a single numerical score. Both stages play roughly equally important roles in real industrial systems; however, how to optimize the model weights for the second stage still lacks systematic study. This paper focuses on optimizing the second stage through auto-tuning technology. Although general auto-tuning systems and solutions - both from established production practices and open-source solutions - can address this problem, they typically require weeks or even months to identify a feasible solution. Such prolonged tuning processes are unacceptable in production environments for recommendation systems, as suboptimal value models can severely degrade user experience. An effective auto-tuning solution is required to identify a viable model within 2-3 days, rather than the extended timelines typically associated with existing approaches. In this paper, we introduce a practical auto-tuning system named HyperZero that addresses these time constraints while effectively solving the unique challenges inherent in modern recommendation systems. Moreover, this framework has the potential to be expanded to broader tuning tasks within recommendation systems.

Paper Structure

This paper contains 25 sections, 14 equations, 9 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Problem Setting
  5. HyperZero: Auto-Tuning System
  6. Data Normalization
  7. Zeroth-Order Optimization (GP + TS) for Multiple Constraints
  8. Gaussian process estimation for Challenge 2(a).
  9. Thompson sampling for Challenge 2(b)
  10. Overall algorithm framework.
  11. Asynchronous parallel exploration
  12. Parallelization
  13. Asynchronous execution
  14. Overall system design
  15. Experiments
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Hyperparameter tuning in common scenarios and recommendation systems respectively.
  • Figure 2: Hourly video view counts across different surfaces from an open-source dataset gao2022kuairand.
  • Figure 3: Workflow of asynchronous parallelization in HyperZero.
  • Figure 4: System design of HyperZero auto-tuning system.
  • Figure 5: Ablation study of HyperZero on synthetic data.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Example 2.1