Intelligent Healthcare Imaging Platform: A VLM-Based Framework for Automated Medical Image Analysis and Clinical Report Generation
Samer Al-Hamadani
TL;DR
This work presents a Vision-Language Model–driven framework for automated, multi-modal medical image analysis that unifies tumor detection and clinical report generation across CT, MRI, X-ray, and ultrasound. It introduces coordinate validation, Gaussian statistical modeling, and a multi-layer visualization suite to enhance spatial precision and interpretability, backed by a Gradio-based user interface for clinical workflow integration. The system attains approximately 80-pixel spatial localization accuracy and supports zero-shot learning to reduce data dependency, while delivering structured reports compatible with healthcare documentation standards. The findings suggest significant potential for improved diagnostic efficiency and decision support, though the authors call for multi-center clinical validation to establish broader clinical adoption.
Abstract
The rapid advancement of artificial intelligence (AI) in healthcare imaging has revolutionized diagnostic medicine and clinical decision-making processes. This work presents an intelligent multimodal framework for medical image analysis that leverages Vision-Language Models (VLMs) in healthcare diagnostics. The framework integrates Google Gemini 2.5 Flash for automated tumor detection and clinical report generation across multiple imaging modalities including CT, MRI, X-ray, and Ultrasound. The system combines visual feature extraction with natural language processing to enable contextual image interpretation, incorporating coordinate verification mechanisms and probabilistic Gaussian modeling for anomaly distribution. Multi-layered visualization techniques generate detailed medical illustrations, overlay comparisons, and statistical representations to enhance clinical confidence, with location measurement achieving 80 pixels average deviation. Result processing utilizes precise prompt engineering and textual analysis to extract structured clinical information while maintaining interpretability. Experimental evaluations demonstrated high performance in anomaly detection across multiple modalities. The system features a user-friendly Gradio interface for clinical workflow integration and demonstrates zero-shot learning capabilities to reduce dependence on large datasets. This framework represents a significant advancement in automated diagnostic support and radiological workflow efficiency, though clinical validation and multi-center evaluation are necessary prior to widespread adoption.