MetaSynth: Multi-Agent Metadata Generation from Implicit Feedback in Black-Box Systems

TL;DR

The paper tackles optimizing meta titles and descriptions in black-box search/ranking environments where explicit labels are unavailable. It introduces MetaSynth, a three-component framework comprising an exemplar library, a constrained generation module, and an evaluator-refinement loop that learns from implicit feedback from top-ranked results. Offline and online experiments on proprietary data and the Amazon dataset show state-of-the-art improvements in NDCG, MRR, and ranking metrics, with significant CTR and click increases in live A/B tests. The approach demonstrates a general paradigm for leveraging weak supervision to optimize content in black-box systems, with potential extensions to other domains and modalities.

Abstract

Meta titles and descriptions strongly shape engagement in search and recommendation platforms, yet optimizing them remains challenging. Search engine ranking models are black box environments, explicit labels are unavailable, and feedback such as click-through rate (CTR) arrives only post-deployment. Existing template, LLM, and retrieval-augmented approaches either lack diversity, hallucinate attributes, or ignore whether candidate phrasing has historically succeeded in ranking. This leaves a gap in directly leveraging implicit signals from observable outcomes. We introduce MetaSynth, a multi-agent retrieval-augmented generation framework that learns from implicit search feedback. MetaSynth builds an exemplar library from top-ranked results, generates candidate snippets conditioned on both product content and exemplars, and iteratively refines outputs via evaluator-generator loops that enforce relevance, promotional strength, and compliance. On both proprietary e-commerce data and the Amazon Reviews corpus, MetaSynth outperforms strong baselines across NDCG, MRR, and rank metrics. Large-scale A/B tests further demonstrate 10.26% CTR and 7.51% clicks. Beyond metadata, this work contributes a general paradigm for optimizing content in black-box systems using implicit signals.

Figures (4)

• Figure 1: An example of how MetaSynth optimizes search engine meta descriptions. The top snippet (pre-optimization) is factual but generic, while the bottom snippet (MetaSynth) emphasizes promotional value, readability, and policy compliance. Such refinements directly impact user engagement and search-driven traffic by producing coherent and persuasive messages.
• Figure 2: MetaSynth Framework: Three main modules to generate and optimize meta titles and descriptions for items, according to seller's provided information, and constraints for better search engine ranking. Left block represents the RAG-based exemplar library creation. Middle block shows the generation module for both meta title and description based on seller provided data and exemplar library. Right block represents the evaluation/refinement loop, where multi evaluator agents evaluate the title/description for their assigned criteria, and if failed to satisfy that criteria, the feedback consolidation agent directs such title/description to re-generation agent along with the provided feedback from rejecting evaluator.
• Figure 3: Two examples showcasing the edits done by evaluation and refinement agents on RAG outputs, to make the description more accurate and engaging, according to seller's provided description.
• Figure 4: Case studies comparing MetaSynth method with three other studies (baseline, DRE, COT), in which MetaSynth outcome ranked first among others in the search engine. Some influential words that might be affected the ranking are demonstrated with bold font. Across four verticals of home décor, furniture, apparel, and tailgating/barware, MetaSynth replaces specification lists with fluent, benefit‑led phrasing while preserving key retrieval tokens. This selective emphasis improves readability and broadens intent matching without keyword stuffing, aligning with the relevance–readability–compliance objectives of the method.