Reliable Inference in Edge-Cloud Model Cascades via Conformal Alignment

TL;DR

Edge predictions must achieve cloud-level conditional coverage with respect to the cloud distribution $p^*(y|x)$. The authors introduce Conformal Alignment-Based (CAb) cascading, which casts edge-to-cloud escalation as a multiple-hypothesis testing problem and uses a learned alignment score to gate edge decisions, guaranteeing that the edge-processed fraction meets the cloud-level conditional coverage on average while controlling deferral and set size. The approach is compatible with arbitrary edge prediction sets, including CP variants, and provides finite-sample FDR-type guarantees via sequential screening and optional stopping. Empirical results on CIFAR-100 and TeleQnA demonstrate that CAb reduces cloud offloading substantially with only modest increases in prediction-set size, offering a practical reliability-cost trade-off for edge-cloud inference.

Abstract

Edge intelligence enables low-latency inference via compact on-device models, but assuring reliability remains challenging. We study edge-cloud cascades that must preserve conditional coverage: whenever the edge returns a prediction set, it should contain the true label with a user-specified probability, as if produced by the cloud model. We formalize conditional coverage with respect to the cloud predictive distribution, and introduce a conformal alignment-based (CAb) cascading mechanism that certifies this property with user control over the risk level. Our method casts escalation from edge to cloud models as a multiple-hypothesis testing (MHT) problem, tailoring conformal alignment (CA) to select which inputs can be safely handled at the edge. The proposed CAb model cascading method yields statistical guarantees on the average fraction of edge decisions that satisfy cloud-level conditional coverage. The procedure applies to arbitrary edge prediction sets, including variants of conformal prediction (CP), and exposes a tunable trade-off among coverage, deferral rate, and set size. Experiments on CIFAR-100 image classification and the TeleQnA question-answering (QA) benchmark show that the proposed CAb cascade maintains the target conditional coverage for edge predictions while substantially reducing offloading to the cloud and incurring modest increases in prediction-set size.

Figures (10)

• Figure 1: In the edge-cloud cascade model under study, the goal is to produce a prediction set that is as reliable as the one produced by the cloud model, while leveraging the edge model for as many inputs as possible.
• Figure 2: Given a batch of test input $\mathcal{D}^{\text{te}}$, the small-scale edge model generates prediction sets that may deviate from the prediction sets that would have been produced by a large-scale cloud model, failing to meet a target reliability requirement. The proposed method, conformal alignment-based (CAb) model cascading, casts the edge-cloud escalation as a multiple-hypothesis testing (MHT) problem, determining when to trust the edge prediction set based on reference data $\mathcal{D}$. CAb controls the fraction of edge-generated prediction sets that satisfy conditional coverage conditions, while minimizing the deferral rate to the cloud model.
• Figure 3: The proposed CAb model cascading adopts a sequential screening methodology to ensure that the edge-processed subset $\mathcal{S}$ satisfies the constraint (\ref{['eq:batch_goal']}). This procedure evaluates inputs in the joint test and validation dataset $\mathcal{D}^{\text{te}} \cup \mathcal{D}^{\text{val}}$ in the order (\ref{['eq:screen_order']}) of increasing estimate alignment score $\hat{C}(x)$. At each step, the CAb method estimates the false discovery proportion (FDP) of the unscreened test inputs based on the unscreened validation data using (\ref{['eq:FDP_estimator']}). The procedure terminates at the earliest step $\ell_{\text{CA}}$ in (\ref{['eq:stopping_rule']}) when the estimated FDP of unscreened test inputs falls below the pre-determined tolerated level $\delta$.
• Figure 4: Average satisfaction rate (left) and normalized inefficiency (right) versus target conditional coverage levels $1-\alpha \in \{0.7, 0.75, 0.8, 0.85, 0.9\}$ on the CIFAR-100 dataset, considering edge-only schemes: highest mass set (HMS) in (\ref{['eq:edge_HDS']}), conformal prediction (CP) in (\ref{['eq:edge_CP']}), and localized conformal prediction (LCP) in (\ref{['eq:edge_LCP']}) with Gaussian kernel bandwidth $h=15$ and $h=20$, respectively.
• Figure 5: Reliability diagram for the edge model, namely WideResNet-40-2 model, on the CIFAR-100 dataset.