Computational Imaging Meets LLMs: Zero-Shot IDH Mutation Prediction in Brain Gliomas
Syed Muqeem Mahmood, Hassan Mohy-ud-Din
TL;DR
Non-invasive IDH mutation prediction in brain gliomas is addressed by a zero-shot framework that fuses imaging-derived semantic attributes and quantitative MRI features with LLM reasoning via prompts to GPT-4o and GPT-5, without fine-tuning. The method uses co-registered 3D mpMRI, segmentation maps, and a standardized JSON representation, evaluated across six public datasets totaling $N=1427$ subjects, with GPT-5 generally outperforming GPT-4o in accuracy and sensitivity. A modular Computational Imaging Toolbox extracts five feature groups and reveals volumetric measures as the strongest predictors, supplemented by imaging markers and basic clinical data. The study demonstrates the feasibility and potential of integrating multimodal imaging analytics with foundation-model reasoning for rapid, generalizable radiogenomic genotyping in neuro-oncology, with code available on GitHub.
Abstract
We present a framework that combines Large Language Models with computational image analytics for non-invasive, zero-shot prediction of IDH mutation status in brain gliomas. For each subject, coregistered multi-parametric MRI scans and multi-class tumor segmentation maps were processed to extract interpretable semantic (visual) attributes and quantitative features, serialized in a standardized JSON file, and used to query GPT 4o and GPT 5 without fine-tuning. We evaluated this framework on six publicly available datasets (N = 1427) and results showcased high accuracy and balanced classification performance across heterogeneous cohorts, even in the absence of manual annotations. GPT 5 outperformed GPT 4o in context-driven phenotype interpretation. Volumetric features emerged as the most important predictors, supplemented by subtype-specific imaging markers and clinical information. Our results demonstrate the potential of integrating LLM-based reasoning with computational image analytics for precise, non-invasive tumor genotyping, advancing diagnostic strategies in neuro-oncology. The code is available at https://github.com/ATPLab-LUMS/CIM-LLM.