Uncertainty-Aware Dynamic Knowledge Graphs for Reliable Question Answering

TL;DR

This work tackles the problem of unreliable QA when evidence is evolving and uncertain by introducing uncertainty-aware dynamic knowledge graphs (KGs) and personalized KGs (PKGs) that update with new patient data. The approach combines dynamic KG construction, explicit confidence estimation with scores $s \in [0,1]$, and confidence-aware retrieval to produce more robust and interpretable QA, demonstrated in healthcare using EHR-derived PKGs. The key contributions include a dynamic PKG construction pipeline, a formal confidence-estimation mechanism yielding calibrated scores, and a comparative QA framework that contrasts baseline and confidence-aware retrieval, validated on the MIMIC-III mortality-prediction task with notable improvements in AUROC and AUPRC. The results indicate that pruning low-confidence evidence reduces hallucinations and improves calibration, suggesting broad applicability of uncertainty-aware dynamic KGs for trustworthy LLM-assisted QA in high-stakes domains.

Abstract

Question answering (QA) systems are increasingly deployed across domains. However, their reliability is undermined when retrieved evidence is incomplete, noisy, or uncertain. Existing knowledge graph (KG) based QA frameworks typically represent facts as static and deterministic, failing to capture the evolving nature of information and the uncertainty inherent in reasoning. We present a demonstration of uncertainty-aware dynamic KGs, a framework that combines (i) dynamic construction of evolving KGs, (ii) confidence scoring and uncertainty-aware retrieval, and (iii) an interactive interface for reliable and interpretable QA. Our system highlights how uncertainty modeling can make QA more robust and transparent by enabling users to explore dynamic graphs, inspect confidence-annotated triples, and compare baseline versus confidence-aware answers. The target users of this demo are clinical data scientists and clinicians, and we instantiate the framework in healthcare: constructing personalized KGs from electronic health records, visualizing uncertainty across patient visits, and evaluating its impact on a mortality prediction task. This use case demonstrates the broader promise of uncertainty-aware dynamic KGs for enhancing QA reliability in high-stakes applications.

Figures (4)

• Figure 1: Uncertainty-aware reasoning via dynamic KGs.
• Figure 2: System Architecture. The pipeline has three stages: Dynamic KG Construction, Confidence-aware QA, and Visualization. During the Construction stage, multiple KG variants are materialized in the Dynamic KG Store (see Sections \ref{['sec:kg_construction']} and \ref{['sec:confidence_estimation']} for definitions). Subsequent QA and Visualization stages retrieve and operate on these stored variants. Solid arrows denote update, while dashed arrows indicate reference.
• Figure 3: Example of clinical QA based on a filtered PKG (patient: red, visit: white, disease: brown, procedure: green, medication: pink). The user can modify the question on the fly and compare the answers generated with different PKGs. In this case, the baseline answer suggests a generic antithrombotic drug based on the diagnoses of the patient. The Filtered KG version, which includes background knowledge and confidence, favors beta blockers citing previous high-confidence use of this drug for the patient.
• Figure 4: Hovering over edges allows for exploring the rationale behind the attributed confidence score, including previous evidence.