To Trust or Not to Trust: On Calibration in ML-based Resource Allocation for Wireless Networks

TL;DR

This work investigates calibration for ML-based outage predictors in a single-user, multi-resource wireless setting. It derives theoretical properties of outage probability under perfect calibration, showing that in the infinite-resource limit the OP equals the conditional expectation $\mathbb{E}[Q^{c}\mid Q^{c} \le q_{th}]$, while with a single resource it equals $\mathbb{E}[Q^{c}]$, and it establishes that post-processing calibration cannot reduce the minimum OP. It also introduces a monotonicity condition on the accuracy–confidence function under which calibration improves OP and validates these findings with simulations using Platt scaling and isotonic regression on Rayleigh Clarke fading channels with an outage-loss–informed predictor. The results guide threshold selection to meet reliability targets (SLA) and demonstrate practical viability via a proof-of-concept, including reproducible code. Overall, the paper provides both theoretical and empirical foundations for trusted, calibrated outage prediction in ML-driven wireless resource allocation.

Abstract

In next-generation communications and networks, machine learning (ML) models are expected to deliver not only accurate predictions but also well-calibrated confidence scores that reflect the true likelihood of correct decisions. This paper studies the calibration performance of an ML-based outage predictor within a single-user, multi-resource allocation framework. We first establish key theoretical properties of this system's outage probability (OP) under perfect calibration. Importantly, we show that as the number of resources grows, the OP of a perfectly calibrated predictor approaches the expected output conditioned on it being below the classification threshold. In contrast, when only one resource is available, the system's OP equals the model's overall expected output. We then derive the OP conditions for a perfectly calibrated predictor. These findings guide the choice of the classification threshold to achieve a desired OP, helping system designers meet specific reliability requirements. We also demonstrate that post-processing calibration cannot improve the system's minimum achievable OP, as it does not introduce new information about future channel states. Additionally, we show that well-calibrated models are part of a broader class of predictors that necessarily improve OP. In particular, we establish a monotonicity condition that the accuracy-confidence function must satisfy for such improvement to occur. To demonstrate these theoretical properties, we conduct a rigorous simulation-based analysis using post-processing calibration techniques: Platt scaling and isotonic regression. As part of this framework, the predictor is trained using an outage loss function specifically designed for this system. Furthermore, this analysis is performed on Rayleigh fading channels with temporal correlation captured by Clarke's 2D model, which accounts for receiver mobility.

Figures (10)

• Figure 1: Reliability diagram showing the accuracy-confidence function $A_j(q)$ (top), and the confidence distribution $F_j(q)$ (bottom).
• Figure 2: Illustration of Theorem \ref{['thm thm2']}, where (a) a predictor is useful at $q_{th} = 0.5$, satisfying \ref{['eq:useful_predictor']}, and (b) a predictor is not useful at the same $q_{th}$.
• Figure 3: System-level overview of the ML-based outage prediction framework. The predictor takes dynamic wireless channel data for resource $j\in \mathcal{R}$ as input and produces uncalibrated confidence estimates. Post-processing calibration yields calibrated confidence values, which are then used for threshold-based resource selection.
• Figure 4: Flowchart illustrating the key steps required to reproduce the proposed calibration-aware ML-based resource allocation framework.
• Figure 5: Histogram-based reliability diagrams of our predictor utilizing OLF and BCE for a 4-resource system with a rate threshold $\gamma_{th}$ of 0.5, using both LSTM and DQN-LSTM model architectures. Specifically, plots are presented for $\mathtt{SNR}$ values of (a) 2 dB and (b) 8 dB, for predictors trained with OLF, and (c) 2 dB and (d) 8 dB, for predictors trained with BCE, each with various calibration methods applied.

Theorems & Definitions (4)

• Theorem 1
• Corollary 1
• Theorem 2
• Theorem 3