Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems

TL;DR

The paper tackles reliable, cost-efficient telecom knowledge-automation by designing a three-tier edge–cloud–expert cascade for LLM-based QA. It introduces a misalignment-cost constrained optimization and a statistically rigorous thresholding method (MHT-ERM) to guarantee finite-sample reliability while minimizing average cost. The approach leverages epistemic-uncertainty and confidence scores from white-box Bayesian ensembles or prompt-based methods for black-box models, and validates the framework on TeleQnA with both conventional and reasoning-enhanced cloud deployments. Results show substantial cost savings over baselines while meeting prescribed misalignment guarantees, with robustness to calibration data size and cloud reasoning budgets, highlighting practical applicability in telecom knowledge systems.

Abstract

Large language models (LLMs) are emerging as key enablers of automation in domains such as telecommunications, assisting with tasks including troubleshooting, standards interpretation, and network optimization. However, their deployment in practice must balance inference cost, latency, and reliability. In this work, we study an edge-cloud-expert cascaded LLM-based knowledge system that supports decision-making through a question-and-answer pipeline. In it, an efficient edge model handles routine queries, a more capable cloud model addresses complex cases, and human experts are involved only when necessary. We define a misalignment-cost constrained optimization problem, aiming to minimize average processing cost, while guaranteeing alignment of automated answers with expert judgments. We propose a statistically rigorous threshold selection method based on multiple hypothesis testing (MHT) for a query processing mechanism based on knowledge and confidence tests. The approach provides finite-sample guarantees on misalignment risk. Experiments on the TeleQnA dataset -- a telecom-specific benchmark -- demonstrate that the proposed method achieves superior cost-efficiency compared to conventional cascaded baselines, while ensuring reliability at prescribed confidence levels.

Figures (7)

• Figure 1: Cascaded edge-cloud-human system: The query is processed by the edge model $M_{\text{edge}}$ if the edge model's epistemic uncertainty $U_{\text{edge}}(x)$ remains within the acceptable level $\epsilon$, while the confidence $C_{\text{edge}}(x)$ exceeds a threshold $\lambda$, i.e., $U_{\text{edge}}(x) < \epsilon$ and $C_{\text{edge}}(x) > \lambda$. Thus, the edge decision $M_{\text{edge}}(x)$ is produced only if the edge model is sufficiently knowledgeable and confident. When the edge epistemic uncertainty condition is not met, and thus the edge model does not have sufficient knowledge to address the query, the input $x$ is forwarded to the cloud model $M_{\text{cloud}}$. Similar knowledge and confidence tests are carried out for the cloud model based on epistemic uncertainty measure $U_{\text{cloud}}(x)$ and confidence measure $C_{\text{cloud}}(x)$. If the cloud model passes the test, i.e., $U_{\text{cloud}}(x) < \epsilon$ and $C_{\text{cloud}}(x) > \lambda$, the cloud decision $M_{\text{cloud}}(x)$ is returned, otherwise, the input $x$ is deferred to a human expert.
• Figure 2: Illustration of the parallel fixed sequence testing MHT step carried out by the proposed MHT-ERM methodology. For each $m$-th sequence corresponding to a value $\epsilon_m$ of the confidence threshold, a pair of thresholds $(\epsilon_m,\lambda_q)$ is tested at each step, starting from $\lambda_Q=1$ and progressively decreasing $q$ through the sequence. The p-value of each pair of thresholds is compared against the risk level $\delta/M$ to assess the reliability of thresholds $(\epsilon_m,\lambda_q)$. All descendants of unreliable thresholds are deemed unreliable.
• Figure 3: Misalignment and corresponding cost for edge-only, cloud-only, and human-only schemes, as well as for the cascading systems designed via C-ERM, MHT-ERM-B, and MHT-ERM. We set the target misalignment risk in (\ref{['constraint']}) to $\alpha=0.3$ (dashed line) and the target reliability in (\ref{['FWER_goal']}) to $1-\delta=0.95$. The colored horizontal lines mark the $1-\delta=0.95$-quantile values of the misalignment rate. Maximal values in misalignment performance and cost performance are reported within the $1.5$ interquartile range (IQR) range box_plot across $200$ independent experiments.
• Figure 4: Misalignment and corresponding cost for the cascading systems with thresholds chosen via C-ERM, MHT-ERM-B, and MHT-ERM under different values of calibration dataset size. We set the target misalignment risk in (\ref{['constraint']}) to $\alpha=0.3$ (dashed line) and target reliability in (\ref{['FWER_goal']}) to $1-\delta=0.95$. The results are averaged over $200$ independent experiments (shaded bar on plots shows one standard deviation on both sides).
• Figure 5: Misalignment and corresponding cost for the cascading systems with thresholds chosen via C-ERM, MHT-ERM-B, and MHT-ERM under different values of misalignment upper bound. We set the target misalignment risk in (\ref{['constraint']}) to $\alpha=0.3$ (dashed line) and target reliability in (\ref{['FWER_goal']}) to $1-\delta=0.95$. The results are averaged over $200$ independent experiments (shaded bar on plots shows one standard deviation on both sides).

Theorems & Definitions (2)

• Proposition 1: Simultaneous alignment guarantees for subset $\Phi^*$
• Corollary 1: Alignment guarantee for the selected thresholds $\phi^*$