Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interactive Segmentation and Report Generation for CT Images

Yannian Gu, Wenhui Lei, Hanyu Chen, Xiaofan Zhang, Shaoting Zhang

TL;DR

To the best knowledge, this work is the first to integrate the interactive segmentation and structured reports in 3D CT medical images and demonstrates the effectiveness of the approach in providing a more comprehensive and reliable reporting system for lesion segmentation and capturing.

Abstract

Automated CT report generation plays a crucial role in improving diagnostic accuracy and clinical workflow efficiency. However, existing methods lack interpretability and impede patient-clinician understanding, while their static nature restricts radiologists from dynamically adjusting assessments during image review. Inspired by interactive segmentation techniques, we propose a novel interactive framework for 3D lesion morphology reporting that seamlessly generates segmentation masks with comprehensive attribute descriptions, enabling clinicians to generate detailed lesion profiles for enhanced diagnostic assessment. To our best knowledge, we are the first to integrate the interactive segmentation and structured reports in 3D CT medical images. Experimental results across 15 lesion types demonstrate the effectiveness of our approach in providing a more comprehensive and reliable reporting system for lesion segmentation and capturing. The source code will be made publicly available following paper acceptance.

Interactive Segmentation and Report Generation for CT Images

TL;DR

To the best knowledge, this work is the first to integrate the interactive segmentation and structured reports in 3D CT medical images and demonstrates the effectiveness of the approach in providing a more comprehensive and reliable reporting system for lesion segmentation and capturing.

Abstract

Automated CT report generation plays a crucial role in improving diagnostic accuracy and clinical workflow efficiency. However, existing methods lack interpretability and impede patient-clinician understanding, while their static nature restricts radiologists from dynamically adjusting assessments during image review. Inspired by interactive segmentation techniques, we propose a novel interactive framework for 3D lesion morphology reporting that seamlessly generates segmentation masks with comprehensive attribute descriptions, enabling clinicians to generate detailed lesion profiles for enhanced diagnostic assessment. To our best knowledge, we are the first to integrate the interactive segmentation and structured reports in 3D CT medical images. Experimental results across 15 lesion types demonstrate the effectiveness of our approach in providing a more comprehensive and reliable reporting system for lesion segmentation and capturing. The source code will be made publicly available following paper acceptance.

Paper Structure

This paper contains 10 sections, 6 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Vision-Prompt Hybrid Token Encode Module
  4. Synergistic Task Integration Decode Module
  5. Training Strategy
  6. Experiments
  7. Dataset Description
  8. Comparision with the SOTA Methods
  9. Interactive Framework Performance
  10. Conclusion

Figures (2)

  • Figure 1: The model architecture. Our framework processes 3D CT images and user-provided points to simultaneously generate lesion segmentation masks and structured attribute descriptions. The model integrates visual tokens, point tokens, initial mask tokens, and initial IOU tokens into a hybrid encoding module, which forms the foundation for both visual and textual outputs. Key innovations include a clustering-based point refinement technique that optimizes the input points through clustering centers, and an inter-task feature synergy mechanism that enhances the performance of both segmentation and attribute description tasks concurrently.
  • Figure 2: Qualitative comparison between our method and SAM-Med3D, showing segmentation progression across click iterations. Red overlays indicate masks.