Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AdNanny: One Reasoning LLM for All Offline Ads Recommendation Tasks

Nan Hu, Han Li, Jimeng Sun, Lu Wang, Fangkai Yang, Bo Qiao, Pu Zhao, David Dai, Mengyu Liu, Yuefeng Zhan, Jianjin Zhang, Weihao Han, Allen Sun, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Denvy Deng, Feng Sun, Qi Zhang

TL;DR

AdNanny tackles the latency constraints of deploying LLMs in online ads by introducing a single, reasoning-centric offline backbone trained on reasoning-augmented data. It unifies multiple Bing Ads offline tasks—relevance labeling, model evaluation, query generation, user profiling, and ad optimization—under one framework, refined through multi-task supervised fine-tuning and RL aligned to downstream metrics. The key contributions are a scalable, centralized backbone that replaces many task-specific models, improved interpretability via explicit reasoning, and production deployment with significant reductions in labeling effort and operational cost. This work demonstrates a practical pathway for large-scale advertising systems to harness LLM reasoning offline while keeping online latency and costs manageable.

Abstract

Large Language Models (LLMs) have shown strong capabilities in Natural Language Understanding and Generation, but deploying them directly in online advertising systems is often impractical due to strict millisecond-level latency constraints. This has motivated the use of LLMs offline to improve retrieval, ranking, and recommendation models. Existing solutions typically fine-tune separate LLMs for individual tasks such as query-ad relevance labeling, keyword-based query generation, and user profiling. This results in redundant models, high maintenance cost, and limited performance gains despite substantial overlap in domain knowledge and reasoning patterns. We introduce AdNanny, a unified reasoning-centric LLM that serves as a shared backbone for offline advertising tasks. AdNanny is obtained by fine-tuning a public 671B-parameter DeepSeek-R1 checkpoint using a scalable training system that supports hybrid dense-MoE parallelism. We construct reasoning-augmented corpora that pair structured supervision with step-by-step natural language explanations. A multi-task supervised fine-tuning stage with adaptive reweighting enables AdNanny to handle diverse labeling and generation tasks in a consistent reasoning format. This is followed by reinforcement learning using downstream advertising metrics to align model behavior with online retrieval and ranking objectives. AdNanny is deployed in production within Bing Ads, where it significantly reduces manual labeling effort and improves accuracy across multiple offline tasks. By consolidating many task-specific models into a single reasoning-centric foundation model, AdNanny provides a scalable and cost-effective solution for large-scale advertising systems.

AdNanny: One Reasoning LLM for All Offline Ads Recommendation Tasks

TL;DR

AdNanny tackles the latency constraints of deploying LLMs in online ads by introducing a single, reasoning-centric offline backbone trained on reasoning-augmented data. It unifies multiple Bing Ads offline tasks—relevance labeling, model evaluation, query generation, user profiling, and ad optimization—under one framework, refined through multi-task supervised fine-tuning and RL aligned to downstream metrics. The key contributions are a scalable, centralized backbone that replaces many task-specific models, improved interpretability via explicit reasoning, and production deployment with significant reductions in labeling effort and operational cost. This work demonstrates a practical pathway for large-scale advertising systems to harness LLM reasoning offline while keeping online latency and costs manageable.

Abstract

Large Language Models (LLMs) have shown strong capabilities in Natural Language Understanding and Generation, but deploying them directly in online advertising systems is often impractical due to strict millisecond-level latency constraints. This has motivated the use of LLMs offline to improve retrieval, ranking, and recommendation models. Existing solutions typically fine-tune separate LLMs for individual tasks such as query-ad relevance labeling, keyword-based query generation, and user profiling. This results in redundant models, high maintenance cost, and limited performance gains despite substantial overlap in domain knowledge and reasoning patterns. We introduce AdNanny, a unified reasoning-centric LLM that serves as a shared backbone for offline advertising tasks. AdNanny is obtained by fine-tuning a public 671B-parameter DeepSeek-R1 checkpoint using a scalable training system that supports hybrid dense-MoE parallelism. We construct reasoning-augmented corpora that pair structured supervision with step-by-step natural language explanations. A multi-task supervised fine-tuning stage with adaptive reweighting enables AdNanny to handle diverse labeling and generation tasks in a consistent reasoning format. This is followed by reinforcement learning using downstream advertising metrics to align model behavior with online retrieval and ranking objectives. AdNanny is deployed in production within Bing Ads, where it significantly reduces manual labeling effort and improves accuracy across multiple offline tasks. By consolidating many task-specific models into a single reasoning-centric foundation model, AdNanny provides a scalable and cost-effective solution for large-scale advertising systems.
Paper Structure (28 sections, 8 equations, 3 figures, 10 tables)

This paper contains 28 sections, 8 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Offline tasks
  3. Data Preparation
  4. Unified reasoning-augmented pipeline
  5. Reasoning generation for representative tasks
  6. Golden-set prompt validation
  7. Rejection sampling and corpus construction
  8. Training Pipeline
  9. Parallelism and Efficiency
  10. Megatron-based parallelism design
  11. Checkpoint conversion and sharding
  12. Distributed optimizer for hybrid dense–MoE layers
  13. Trainer verification
  14. Multi-Task Supervised Fine-Tuning
  15. Multi-task objective with reasoning-augmented targets
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Unified reasoning-augmented data preparation pipeline. DeepSeek-R1 first uses task-specific reasoning prompts to generate label+reasoning candidates (Reasoning Generation); these reasoning prompts are then validated and refined on golden sets (Golden-set Prompt Validation), before being applied at scale with rejection sampling to construct the high-quality corpus used to fine-tune AdNanny.
  • Figure 2: DeepSeek-R1 Parallelism Design.
  • Figure 3: Checkpoint conversion and sharding.