Multivariate Latent Recalibration for Conditional Normalizing Flows

TL;DR

This work tackles the challenge of calibrating multivariate conditional distributions learned by flexible generators. It introduces latent recalibration (LR), a post-hoc method that operates in the latent space of conditional normalizing flows to achieve multivariate latent calibration and produce an explicit recalibrated PDF. LR delivers finite-sample calibration guarantees and maintains computational efficiency, outperforming uncalibrated baselines in latent calibration and often achieving better negative log-likelihood on tabular and high-dimensional image data. By connecting LR with conformal prediction and HDR concepts, the approach offers practical uncertainty quantification with actionable density estimates for decision-making tasks across domains.

Abstract

Reliably characterizing the full conditional distribution of a multivariate response variable given a set of covariates is crucial for trustworthy decision-making. However, misspecified or miscalibrated multivariate models may yield a poor approximation of the joint distribution of the response variables, leading to unreliable predictions and suboptimal decisions. Furthermore, standard recalibration methods are primarily limited to univariate settings, while conformal prediction techniques, despite generating multivariate prediction regions with coverage guarantees, do not provide a full probability density function. We address this gap by first introducing a novel notion of latent calibration, which assesses probabilistic calibration in the latent space of a conditional normalizing flow. Second, we propose latent recalibration (LR), a novel post-hoc model recalibration method that learns a transformation of the latent space with finite-sample bounds on latent calibration. Unlike existing methods, LR produces a recalibrated distribution with an explicit multivariate density function while remaining computationally efficient. Extensive experiments on both tabular and image datasets show that LR consistently improves latent calibration error and the negative log-likelihood of the recalibrated models.

Figures (15)

• Figure 1: Illustration of LR for a bivariate output. The first column shows the latent distribution, the second column displays the predictive PDF, and the third and fourth columns show reliability diagrams for latent and HDR calibration, respectively. The first row corresponds to an uncalibrated NF, and the second is the same model after LR. Calibration points and their projections in the latent space are shown in blue. The PDF for both the latent distribution and the predictive distribution is shown in orange. Level sets of the PIT of the latent norm at levels 0.01, 0.1, 0.5, and 0.9 are indicated with black contours in the second column, and their corresponding preimages are shown in the first column. LR improves both latent calibration (third column) and HDR calibration (fourth column). Additional prediction examples on real-world datasets are presented in \ref{['sec:examples_predictions']}.
• Figure 2: Relative NLL and ES on datasets sorted by size, using a convex potential flow model.
• Figure 3: L-ECE and HDR-ECE on datasets sorted by size, using a convex potential flow model.
• Figure 4: Density estimation using KDE with a Gamma kernel.
• Figure 5: Density estimation using a rational quadratic spline.

Theorems & Definitions (3)

• Definition 1
• proposition 1
• proof