Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Augmented Intervenable Multimodal Adaptor for Post-operative Complication Prediction in Lung Cancer Surgery

Shubham Pandey, Bhavin Jawade, Srirangaraj Setlur, Venu Govindaraju, Kenneth Seastedt

TL;DR

MIRACLE addresses the challenge of predicting postoperative complications in lung cancer surgery by unifying preoperative clinical data, radiomic CT biomarkers, and LLM-generated, evidence-grounded explanations into a single, interpretable pipeline. It employs a hyperspherical fusion of modality-specific embeddings via Bayesian MLP encoders and a frozen remark encoder, with a clinician-intervenable, text-grounded explanation channel that can be edited in real time to refine risk predictions, all trained with focal loss to handle class imbalance. On the real-world POC-L dataset (3,094 patients), MIRACLE with OpenBioLLM-70B achieves an AUC of $81.04\%$ and a maximum TPR of $81.31\%$ at $\text{FPR}=0.3$, outperforming classical ML baselines, standalone LLM predictors, and human surgeons, while providing actionable remarks tied to patient-specific factors. The work demonstrates the value of integrating multimodal data with language-based explanations and clinician intervention to create a practical, transparent decision-support tool for perioperative planning, with limitations including demographic bias and the need for multi-institutional validation to ensure generalizability and safety.

Abstract

Postoperative complications remain a critical concern in clinical practice, adversely affecting patient outcomes and contributing to rising healthcare costs. We present MIRACLE, a deep learning architecture for prediction of risk of postoperative complications in lung cancer surgery by integrating preoperative clinical and radiological data. MIRACLE employs a hyperspherical embedding space fusion of heterogeneous inputs, enabling the extraction of robust, discriminative features from both structured clinical records and high-dimensional radiological images. To enhance transparency of prediction and clinical utility, we incorporate an interventional deep learning module in MIRACLE, that not only refines predictions but also provides interpretable and actionable insights, allowing domain experts to interactively adjust recommendations based on clinical expertise. We validate our approach on POC-L, a real-world dataset comprising 3,094 lung cancer patients who underwent surgery at Roswell Park Comprehensive Cancer Center. Our results demonstrate that MIRACLE outperforms various traditional machine learning models and contemporary large language models (LLM) variants alone, for personalized and explainable postoperative risk management.

LLM Augmented Intervenable Multimodal Adaptor for Post-operative Complication Prediction in Lung Cancer Surgery

TL;DR

MIRACLE addresses the challenge of predicting postoperative complications in lung cancer surgery by unifying preoperative clinical data, radiomic CT biomarkers, and LLM-generated, evidence-grounded explanations into a single, interpretable pipeline. It employs a hyperspherical fusion of modality-specific embeddings via Bayesian MLP encoders and a frozen remark encoder, with a clinician-intervenable, text-grounded explanation channel that can be edited in real time to refine risk predictions, all trained with focal loss to handle class imbalance. On the real-world POC-L dataset (3,094 patients), MIRACLE with OpenBioLLM-70B achieves an AUC of and a maximum TPR of at , outperforming classical ML baselines, standalone LLM predictors, and human surgeons, while providing actionable remarks tied to patient-specific factors. The work demonstrates the value of integrating multimodal data with language-based explanations and clinician intervention to create a practical, transparent decision-support tool for perioperative planning, with limitations including demographic bias and the need for multi-institutional validation to ensure generalizability and safety.

Abstract

Postoperative complications remain a critical concern in clinical practice, adversely affecting patient outcomes and contributing to rising healthcare costs. We present MIRACLE, a deep learning architecture for prediction of risk of postoperative complications in lung cancer surgery by integrating preoperative clinical and radiological data. MIRACLE employs a hyperspherical embedding space fusion of heterogeneous inputs, enabling the extraction of robust, discriminative features from both structured clinical records and high-dimensional radiological images. To enhance transparency of prediction and clinical utility, we incorporate an interventional deep learning module in MIRACLE, that not only refines predictions but also provides interpretable and actionable insights, allowing domain experts to interactively adjust recommendations based on clinical expertise. We validate our approach on POC-L, a real-world dataset comprising 3,094 lung cancer patients who underwent surgery at Roswell Park Comprehensive Cancer Center. Our results demonstrate that MIRACLE outperforms various traditional machine learning models and contemporary large language models (LLM) variants alone, for personalized and explainable postoperative risk management.
Paper Structure (29 sections, 9 equations, 2 figures, 2 tables)

This paper contains 29 sections, 9 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Problem Definition
  5. MIRACLE
  6. Remarks Generation Approach:
  7. Training Strategy:
  8. Inference:
  9. Clinician Intervention:
  10. Experiments
  11. Dataset and Preprocessing
  12. Evaluation Metrics
  13. LLMs and Remark Generation
  14. Baselines
  15. Classical Machine‐Learning Methods
  16. ...and 14 more sections

Figures (2)

  • Figure 1: Illustration depicting MIRACLE, our intervenable Deep Learning architecture that combines clinical and radiological features, along with providing surgeons with a point of intervention and explainability. (Best viewed digitally)
  • Figure 2: (a) Receiver Operating Curves for MIRACLE alongside all the baseline models. (b) Distribution of alignment between LLM generated remarks and surgeon‐provided remarks. (c) Confusion Matrix for the surgeons' performance on POC-L test split. (Best Viewed Digitally)