Translating Imaging to Genomics: Leveraging Transformers for Predictive Modeling
Aiman Farooq, Deepak Mishra, Santanu Chaudhury
TL;DR
This work tackles the problem of non-invasively predicting genome-wide expression from standard CT/MRI imaging to support precision oncology. It leverages a transformer-based encoder (CNN front end followed by an 8-layer Transformer via TransUNet) to learn patient-level embeddings and predict RNA-Seq profiles using a mean-squared-error objective, evaluated on TCIA/TCGA data across GBM, LGG, LUAD, and BRCA. The study demonstrates significant imaging-genomics associations for thousands of genes, though fewer than those achieved by WSI-based methods, highlighting both the potential and current limits of non-invasive radiogenomics. Overall, the approach provides a feasible pathway to bedside-accessible genomic profiling from routine imaging, motivating further refinement and integration into clinical workflows.
Abstract
In this study, we present a novel approach for predicting genomic information from medical imaging modalities using a transformer-based model. We aim to bridge the gap between imaging and genomics data by leveraging transformer networks, allowing for accurate genomic profile predictions from CT/MRI images. Presently most studies rely on the use of whole slide images (WSI) for the association, which are obtained via invasive methodologies. We propose using only available CT/MRI images to predict genomic sequences. Our transformer based approach is able to efficiently generate associations between multiple sequences based on CT/MRI images alone. This work paves the way for the use of non-invasive imaging modalities for precise and personalized healthcare, allowing for a better understanding of diseases and treatment.