Deep Ensemble Shape Calibration: Multi-Field Post-hoc Calibration in Online Advertising

TL;DR

This work tackles calibration of predicted probabilities for CTR/CVR in online advertising, focusing on multi-field calibration where field-wise biases and distributional shapes vary widely. It introduces Deep Ensemble Shape Calibration (DESC), decomposing calibration into a shape component and a value component, and builds a novel framework that uses predefined basis calibration functions, per-field shape allocation, and a global shape ensemble to harmonize across fields, plus a global field value calibrator with a softmax-derived global shape attention. DESC achieves superior calibration accuracy on public and industrial data, while maintaining ranking performance, and delivers substantial online gains (e.g., +2.5% CVR and +4.0% GMV). The approach advances practical calibration in high-cardinality, sparse settings and demonstrates strong data-utilization efficiency through shared bases and cross-field attention, making it appealing for deployment in large-scale online advertising systems.

Abstract

In the e-commerce advertising scenario, estimating the true probabilities (known as a calibrated estimate) on Click-Through Rate (CTR) and Conversion Rate (CVR) is critical. Previous research has introduced numerous solutions for addressing the calibration problem. These methods typically involve the training of calibrators using a validation set and subsequently applying these calibrators to correct the original estimated values during online inference. However, what sets e-commerce advertising scenarios apart is the challenge of multi-field calibration. Multi-field calibration requires achieving calibration in each field. In order to achieve multi-field calibration, it is necessary to have a strong data utilization ability. Because the quantity of pCTR specified range for a single field-value (such as user ID and item ID) sample is relatively small, this makes the calibrator more difficult to train. However, existing methods have difficulty effectively addressing these issues. To solve these problems, we propose a new method named Deep Ensemble Shape Calibration (DESC). In terms of business understanding and interpretability, we decompose multi-field calibration into value calibration and shape calibration. We introduce innovative basis calibration functions, which enhance both function expression capabilities and data utilization by combining these basis calibration functions. A significant advancement lies in the development of an allocator capable of allocating the most suitable calibrators to different estimation error distributions within diverse fields and values. We achieve significant improvements in both public and industrial datasets. In online experiments, we observe a +2.5% increase in CVR and +4.0% in GMV (Gross Merchandise Volume). Our code is now available at: https://github.com/HaoYang0123/DESC.

Figures (9)

• Figure 1: Examples to show the shape miscalibration and value miscalibration
• Figure 2: Examples to show significant variations in value miscalibration among different fields and values.
• Figure 3: Examples to show significant variations in shape miscalibration among different values within the same field.
• Figure 4: (a) Overall architecture of DESC, its input includes the non-calibrated pCTR and the original fields, and the final output is the calibrated pCTR. It is end-to-end trainable. (b) Single Field Shape Calibrator takes one field and the non-calibrated pCTR as inputs and outputs the calibrated shape score for this field.
• Figure 5: Complex shape can be composed of simple shapes.