A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation

TL;DR

This work tackles the scarcity and variability of renal CT reports by proposing a clinically informed, two-stage AI framework. Stage 1 detects eight structured renal features from 2D CT slices, while Stage 2 uses a vision-language model to generate free-text radiology reports conditioned on both the image and the detected features. On UF Health data, this approach improves clinical fidelity, with feature extraction from generated reports reaching high accuracy and METEOR/BLEU/ROUGE scores indicating credible text generation, though fidelity remains sensitive to data size and prompt design. The study highlights the importance of domain-specific evaluation and points to future work in extending to 3D volumes, improving feature fidelity, and expanding datasets for robust real-world deployment.

Abstract

Objective Renal cancer is a common malignancy and a major cause of cancer-related deaths. Computed tomography (CT) is central to early detection, staging, and treatment planning. However, the growing CT workload increases radiologists' burden and risks incomplete documentation. Automatically generating accurate reports remains challenging because it requires integrating visual interpretation with clinical reasoning. Advances in artificial intelligence (AI), especially large language and vision-language models, offer potential to reduce workload and enhance diagnostic quality. Methods We propose a clinically informed, two-stage framework for automatic renal CT report generation. In Stage 1, a multi-task learning model detects structured clinical features from each 2D image. In Stage 2, a vision-language model generates free-text reports conditioned on the image and the detected features. To evaluate clinical fidelity, generated clinical features are extracted from the reports and compared with expert-annotated ground truth. Results Experiments on an expert-labeled dataset show that incorporating detected features improves both report quality and clinical accuracy. The model achieved an average AUC of 0.75 for key imaging features and a METEOR score of 0.33, demonstrating higher clinical consistency and fewer template-driven errors. Conclusion Linking structured feature detection with conditioned report generation provides a clinically grounded approach to integrate structured prediction and narrative drafting for renal CT reporting. This method enhances interpretability and clinical faithfulness, underscoring the value of domain-relevant evaluation metrics for medical AI development.

Figures (4)

• Figure 1: Dataset curation and model framework. (a) From expert renal CT radiology reports we derive structured clinical (abnormality) features—position, size, exophytic growth, attenuation, enhancement, cyst, mass, tumor—and a reference slice index; these align each 2D CT slice with labels. (b) From a 3D CT volume, we retrieve the labeled slice and apply intensity clipping to form the 2D input image. (c) Two-stage pipeline: Stage 1 detects clinical features from the image; Stage 2 conditions an LLM-based report generator on the image and the detected features via a structured prompt to produce a narrative clinical report.
• Figure 2: Workflow for filtering renal CT reports and scans. Reports were filtered based on renal-related CPT-9 codes, scan availability, and explicit references to individual CT slices. Reports referencing non-coronal slices or lacking abnormalities were excluded. Some reports contained multiple slice references, resulting in 130 manual annotations from 97 patients.
• Figure 3: Comparison of model-generated radiology reports and extracted features under different input modalities (case 1).
• Figure 4: Comparison of model-generated radiology reports and corresponding extracted features across different input modalities (case 2).