MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising

TL;DR

This work tackles miscalibration in online advertising probability estimates by proposing MCNet, a hybrid post-hoc calibration framework that learns expressive monotone calibration functions (MCF) while being context-aware and field-balanced. By combining an integration-based monotone mapping with an embedding-based context model and two regularizers, MCNet achieves superior calibration quality (PCOC, F-RCE) while preserving ranking (AUC) across diverse datasets. The approach demonstrates improved cross-field fairness and robust performance on CTR and CVR tasks, supported by extensive ablations and function analyses. The result is a practically impactful calibration method for real-time advertising systems, enabling better-aligned probabilities and fairer field-wide exposure.

Abstract

In online advertising, uncertainty calibration aims to adjust a ranking model's probability predictions to better approximate the true likelihood of an event, e.g., a click or a conversion. However, existing calibration approaches may lack the ability to effectively model complex nonlinear relations, consider context features, and achieve balanced performance across different data subsets. To tackle these challenges, we introduce a novel model called Monotonic Calibration Networks, featuring three key designs: a monotonic calibration function (MCF), an order-preserving regularizer, and a field-balance regularizer. The nonlinear MCF is capable of naturally modeling and universally approximating the intricate relations between uncalibrated predictions and the posterior probabilities, thus being much more expressive than existing methods. MCF can also integrate context features using a flexible model architecture, thereby achieving context awareness. The order-preserving and field-balance regularizers promote the monotonic relationship between adjacent bins and the balanced calibration performance on data subsets, respectively. Experimental results on both public and industrial datasets demonstrate the superior performance of our method in generating well-calibrated probability predictions.

Figures (5)

• Figure 1: Illustration of different calibration functions.
• Figure 2: Model architecture of MCNet. MCNet jointly models the uncalibrated score and the context feature to learn a monotonic calibration function. Given a specific context feature (e.g., context 1 and 2), MCNet generates the calibrated probabilities that are context-adaptive and monotonically increasing with the corresponding uncalibrated probabilities.
• Figure 3: Visualization of calibration functions.
• Figure 4: Model architecture with an additional auxiliary network.
• Figure 5: Calibration metrics across bin numbers, learning rates, and balance coefficients $\beta$.

Theorems & Definitions (2)

• theorem 1: Expressiveness
• theorem 1: Expressiveness