MAC: A Conversion Rate Prediction Benchmark Featuring Labels Under Multiple Attribution Mechanisms

TL;DR

The Multi-Attribution Benchmark (MAC) is established, the first public CVR dataset featuring labels from multiple attribution mechanisms, and Mixture of Asymmetric Experts (MoAE), an effective MAL approach incorporating multi-attribution knowledge learning and main task-centric knowledge utilization is proposed.

Abstract

Multi-attribution learning (MAL), which enhances model performance by learning from conversion labels yielded by multiple attribution mechanisms, has emerged as a promising learning paradigm for conversion rate (CVR) prediction. However, the conversion labels in public CVR datasets are generated by a single attribution mechanism, hindering the development of MAL approaches. To address this data gap, we establish the Multi-Attribution Benchmark (MAC), the first public CVR dataset featuring labels from multiple attribution mechanisms. Besides, to promote reproducible research on MAL, we develop PyMAL, an open-source library covering a wide array of baseline methods. We conduct comprehensive experimental analyses on MAC and reveal three key insights: (1) MAL brings consistent performance gains across different attribution settings, especially for users featuring long conversion paths. (2) The performance growth scales up with objective complexity in most settings; however, when predicting first-click conversion targets, simply adding auxiliary objectives is counterproductive, underscoring the necessity of careful selection of auxiliary objectives. (3) Two architectural design principles are paramount: first, to fully learn the multi-attribution knowledge, and second, to fully leverage this knowledge to serve the main task. Motivated by these findings, we propose Mixture of Asymmetric Experts (MoAE), an effective MAL approach incorporating multi-attribution knowledge learning and main task-centric knowledge utilization. Experiments on MAC show that MoAE substantially surpasses the existing state-of-the-art MAL method. We believe that our benchmark and insights will foster future research in the MAL field. Our MAC benchmark and the PyMAL algorithm library are publicly available at https://github.com/alimama-tech/PyMAL.

Figures (5)

• Figure 1: Illustration of single-attribution learning and multi-attribution learning (MAL) for CVR prediction. Our work paves the way for studying MAL by providing the first public benchmark, systematic evaluation, and valuable insights.
• Figure 2: The four conversion attribution mechanisms covered in our MAC benchmark.
• Figure 3: Illustration of three families of baselines. A is the primary target under the target attribution mechanism, and B is an auxiliary target from other attribution mechanisms.
• Figure 4: Information flow comparison between cutting-edge baselines (PLE and NATAL) and our MoAE model. A denotes the main target under the target attribution mechanism, and B is an auxiliary target from other attribution mechanisms.
• Figure 5: The GAUC growth of users grouped by conversion path complexity.