CLIN-LLM: A Safety-Constrained Hybrid Framework for Clinical Diagnosis and Treatment Generation

TL;DR

CLIN-LLM addresses the safety and grounding gaps of clinical LLMs by integrating uncertainty-aware diagnosis with retrieval-grounded treatment generation and post-generation safety checks. The two-stage pipeline combines BioBERT-based classification (with Monte Carlo Dropout and Focal Loss), Biomedical Sentence-BERT retrieval over MedDialog, and FLAN-T5 generation, all wrapped with RxNorm and antibiotic stewardship controls. Empirical results show 98% diagnostic accuracy, 78% Top-5 treatment retrieval precision, a clinician-rated validity of 4.2/5, and a 67% reduction in unsafe antibiotic suggestions, with 18% of low-confidence cases routed to expert review. These findings demonstrate robustness, interpretability, and clinical safety alignment, supporting deployment in resource-limited settings and guiding future expansions into imaging, multilingual support, and live clinical validation.

Abstract

Accurate symptom-to-disease classification and clinically grounded treatment recommendations remain challenging, particularly in heterogeneous patient settings with high diagnostic risk. Existing large language model (LLM)-based systems often lack medical grounding and fail to quantify uncertainty, resulting in unsafe outputs. We propose CLIN-LLM, a safety-constrained hybrid pipeline that integrates multimodal patient encoding, uncertainty-calibrated disease classification, and retrieval-augmented treatment generation. The framework fine-tunes BioBERT on 1,200 clinical cases from the Symptom2Disease dataset and incorporates Focal Loss with Monte Carlo Dropout to enable confidence-aware predictions from free-text symptoms and structured vitals. Low-certainty cases (18%) are automatically flagged for expert review, ensuring human oversight. For treatment generation, CLIN-LLM employs Biomedical Sentence-BERT to retrieve top-k relevant dialogues from the 260,000-sample MedDialog corpus. The retrieved evidence and patient context are fed into a fine-tuned FLAN-T5 model for personalized treatment generation, followed by post-processing with RxNorm for antibiotic stewardship and drug-drug interaction (DDI) screening. CLIN-LLM achieves 98% accuracy and F1 score, outperforming ClinicalBERT by 7.1% (p < 0.001), with 78% top-5 retrieval precision and a clinician-rated validity of 4.2 out of 5. Unsafe antibiotic suggestions are reduced by 67% compared to GPT-5. These results demonstrate CLIN-LLM's robustness, interpretability, and clinical safety alignment. The proposed system provides a deployable, human-in-the-loop decision support framework for resource-limited healthcare environments. Future work includes integrating imaging and lab data, multilingual extensions, and clinical trial validation.

Figures (9)

• Figure 1: CLIN-LLM Framework: A two-stage pipeline comprising uncertainty-aware diagnosis via BioBERT+MCD and safety-filtered RAG-based treatment generation.
• Figure 2: Distribution of 24 disease classes in Symptom2Disease with 80/20 train-validation stratification.
• Figure 3: Architecture of the Uncertainty-Aware Classification Module. BioBERT encodes free-text symptoms while a parallel MLP processes vitals. Monte Carlo Dropout allows multiple stochastic inferences, providing both class probabilities and uncertainty estimates. Low-confidence predictions are flagged for expert review to improve safety.
• Figure 4: Fine-Tuned Classification Model Training Metrics over 10 Epochs.
• Figure 5: Confusion matrix for CLIN-LLM predictions on the Symptoms2Disease dataset (24 classes, 10 samples each). The matrix shows strong diagonal dominance with minimal misclassification, reflecting high classification performance of 98% accuracy, precision, recall, and F1-score.