Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Oncology Agent for IDH1 Mutation Prediction in Low-Grade Glioma

Hafsa Akebli, Adam Shephard, Vincenzo Della Mea, Nasir Rajpoot

TL;DR

The paper presents a Multimodal Oncology Agent (MOA) that combines a TITAN-based histology tool with autonomous reasoning over structured clinical/genomic inputs and external biomedical sources (PubMed, Google, OncoKB) to predict IDH1 mutation status in low-grade glioma. On a TCGA-LGG cohort of 488 cases, MOA alone improves over clinical baselines, and integrating histology features yields the best performance (F1 up to 0.912, accuracy 0.915). The results demonstrate that external knowledge and histology furnish complementary mutation-relevant information, enabling accurate predictions while handling missing data. Future directions include extending modality coverage to radiology/pathology reports and exploring open-weight language models for the agent’s reasoning and tool orchestration.

Abstract

Low-grade gliomas frequently present IDH1 mutations that define clinically distinct subgroups with specific prognostic and therapeutic implications. This work introduces a Multimodal Oncology Agent (MOA) integrating a histology tool based on the TITAN foundation model for IDH1 mutation prediction in low-grade glioma, combined with reasoning over structured clinical and genomic inputs through PubMed, Google Search, and OncoKB. MOA reports were quantitatively evaluated on 488 patients from the TCGA-LGG cohort against clinical and histology baselines. MOA without the histology tool outperformed the clinical baseline, achieving an F1-score of 0.826 compared to 0.798. When fused with histology features, MOA reached the highest performance with an F1-score of 0.912, exceeding both the histology baseline at 0.894 and the fused histology-clinical baseline at 0.897. These results demonstrate that the proposed agent captures complementary mutation-relevant information enriched through external biomedical sources, enabling accurate IDH1 mutation prediction.

Multimodal Oncology Agent for IDH1 Mutation Prediction in Low-Grade Glioma

TL;DR

The paper presents a Multimodal Oncology Agent (MOA) that combines a TITAN-based histology tool with autonomous reasoning over structured clinical/genomic inputs and external biomedical sources (PubMed, Google, OncoKB) to predict IDH1 mutation status in low-grade glioma. On a TCGA-LGG cohort of 488 cases, MOA alone improves over clinical baselines, and integrating histology features yields the best performance (F1 up to 0.912, accuracy 0.915). The results demonstrate that external knowledge and histology furnish complementary mutation-relevant information, enabling accurate predictions while handling missing data. Future directions include extending modality coverage to radiology/pathology reports and exploring open-weight language models for the agent’s reasoning and tool orchestration.

Abstract

Low-grade gliomas frequently present IDH1 mutations that define clinically distinct subgroups with specific prognostic and therapeutic implications. This work introduces a Multimodal Oncology Agent (MOA) integrating a histology tool based on the TITAN foundation model for IDH1 mutation prediction in low-grade glioma, combined with reasoning over structured clinical and genomic inputs through PubMed, Google Search, and OncoKB. MOA reports were quantitatively evaluated on 488 patients from the TCGA-LGG cohort against clinical and histology baselines. MOA without the histology tool outperformed the clinical baseline, achieving an F1-score of 0.826 compared to 0.798. When fused with histology features, MOA reached the highest performance with an F1-score of 0.912, exceeding both the histology baseline at 0.894 and the fused histology-clinical baseline at 0.897. These results demonstrate that the proposed agent captures complementary mutation-relevant information enriched through external biomedical sources, enabling accurate IDH1 mutation prediction.

Paper Structure

This paper contains 15 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. The Dataset
  4. Histology Tool for IDH1 Status Prediction
  5. MOA Configuration for IDH1 Status Prediction
  6. MOA Reports for Quantitative Evaluation
  7. Report generation
  8. Report Embedding and Evaluation
  9. Integrated Evaluation of the Complete Agent
  10. Experiments and Results
  11. Implementation details
  12. Agent Performance and Component Analysis
  13. Conclusion and Future Directions
  14. Compliance with Ethical Standards
  15. Acknowledgements

Figures (2)

  • Figure 1: Demonstration of the MOA framework on a patient case. The GPT-4 Assistant autonomously selects tools based on the clinical context and query, retrieves evidence from OncoKB, PubMed, Google, and the histology tool for IDH1 mutation prediction, and integrates all outputs through RAG reasoning. The synthesized findings are compiled into an MOA report.
  • Figure 2: Integrated evaluation of the MOA for IDH1 mutation prediction. Slide embeddings are concatenated with MOA report embeddings (excluding the histology tool), obtained using a sentence transformer. The fused vector is classified by a 4-layer to predict the IDH1 mutation.