Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CTIS-QA: Clinical Template-Informed Slide-level Question Answering for Pathology

Hao Lu, Ziniu Qian, Yifu Li, Yang Zhou, Bingzheng Wei, Yan Xu

TL;DR

This work tackles the lack of clinically grounded evaluation in whole-slide image (WSI) QA by introducing a Clinical Pathology Report Template (CPRT) guided by CAP Cancer Protocols to systematically extract diagnostic elements from pathology reports. It constructs CTIS-Align (80k slide–description QA pairs) for vision–language alignment and CTIS-Bench (14,879 QA pairs from 977 WSIs) with clinically meaningful questions, ensuring QA tasks reflect real diagnostic workflows. The CTIS-QA model adopts a dual-stream visual encoder that combines a clustering-based global representation with an attention-driven local patch perception module, trained in a two-stage regime using CTIS-Align and CTIS-Bench. Across WSI-VQA and CTIS-Bench benchmarks, CTIS-QA achieves state-of-the-art performance, particularly in complex subtype and receptor-status tasks, demonstrating the value of clinically informed, template-based multimodal learning for pathology. The work lays groundwork for extending template-guided pipelines to other cancers and incorporating temporal pathology data to enhance clinical utility.

Abstract

In this paper, we introduce a clinical diagnosis template-based pipeline to systematically collect and structure pathological information. In collaboration with pathologists and guided by the the College of American Pathologists (CAP) Cancer Protocols, we design a Clinical Pathology Report Template (CPRT) that ensures comprehensive and standardized extraction of diagnostic elements from pathology reports. We validate the effectiveness of our pipeline on TCGA-BRCA. First, we extract pathological features from reports using CPRT. These features are then used to build CTIS-Align, a dataset of 80k slide-description pairs from 804 WSIs for vision-language alignment training, and CTIS-Bench, a rigorously curated VQA benchmark comprising 977 WSIs and 14,879 question-answer pairs. CTIS-Bench emphasizes clinically grounded, closed-ended questions (e.g., tumor grade, receptor status) that reflect real diagnostic workflows, minimize non-visual reasoning, and require genuine slide understanding. We further propose CTIS-QA, a Slide-level Question Answering model, featuring a dual-stream architecture that mimics pathologists' diagnostic approach. One stream captures global slide-level context via clustering-based feature aggregation, while the other focuses on salient local regions through attention-guided patch perception module. Extensive experiments on WSI-VQA, CTIS-Bench, and slide-level diagnostic tasks show that CTIS-QA consistently outperforms existing state-of-the-art models across multiple metrics. Code and data are available at https://github.com/HLSvois/CTIS-QA.

CTIS-QA: Clinical Template-Informed Slide-level Question Answering for Pathology

TL;DR

This work tackles the lack of clinically grounded evaluation in whole-slide image (WSI) QA by introducing a Clinical Pathology Report Template (CPRT) guided by CAP Cancer Protocols to systematically extract diagnostic elements from pathology reports. It constructs CTIS-Align (80k slide–description QA pairs) for vision–language alignment and CTIS-Bench (14,879 QA pairs from 977 WSIs) with clinically meaningful questions, ensuring QA tasks reflect real diagnostic workflows. The CTIS-QA model adopts a dual-stream visual encoder that combines a clustering-based global representation with an attention-driven local patch perception module, trained in a two-stage regime using CTIS-Align and CTIS-Bench. Across WSI-VQA and CTIS-Bench benchmarks, CTIS-QA achieves state-of-the-art performance, particularly in complex subtype and receptor-status tasks, demonstrating the value of clinically informed, template-based multimodal learning for pathology. The work lays groundwork for extending template-guided pipelines to other cancers and incorporating temporal pathology data to enhance clinical utility.

Abstract

In this paper, we introduce a clinical diagnosis template-based pipeline to systematically collect and structure pathological information. In collaboration with pathologists and guided by the the College of American Pathologists (CAP) Cancer Protocols, we design a Clinical Pathology Report Template (CPRT) that ensures comprehensive and standardized extraction of diagnostic elements from pathology reports. We validate the effectiveness of our pipeline on TCGA-BRCA. First, we extract pathological features from reports using CPRT. These features are then used to build CTIS-Align, a dataset of 80k slide-description pairs from 804 WSIs for vision-language alignment training, and CTIS-Bench, a rigorously curated VQA benchmark comprising 977 WSIs and 14,879 question-answer pairs. CTIS-Bench emphasizes clinically grounded, closed-ended questions (e.g., tumor grade, receptor status) that reflect real diagnostic workflows, minimize non-visual reasoning, and require genuine slide understanding. We further propose CTIS-QA, a Slide-level Question Answering model, featuring a dual-stream architecture that mimics pathologists' diagnostic approach. One stream captures global slide-level context via clustering-based feature aggregation, while the other focuses on salient local regions through attention-guided patch perception module. Extensive experiments on WSI-VQA, CTIS-Bench, and slide-level diagnostic tasks show that CTIS-QA consistently outperforms existing state-of-the-art models across multiple metrics. Code and data are available at https://github.com/HLSvois/CTIS-QA.
Paper Structure (15 sections, 3 equations, 5 figures, 3 tables)

This paper contains 15 sections, 3 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Data Construction
  3. Clinical Pathology Report Template
  4. Pathology Features Curation
  5. CTIS-Align Dataset Construction
  6. CTIS-Bench Dataset Construction
  7. Method
  8. Dual-stream Visual Encoder
  9. Patch Perception Module
  10. Experiments
  11. Datasets
  12. Implementation Details
  13. Comparisons
  14. Ablation Study
  15. Conclusion

Figures (5)

  • Figure 1: Comparison between existing VQA datasets and CPRT-generated QA pairs. Left: Failure cases from existing VQA datasets. Right: CPRT-generated QA pairs demonstrating comprehensive coverage of essential pathological features following standardized clinical protocols, meeting the systematic requirements of real diagnostic workflows.
  • Figure 2: Illustration of Data Construction Pipeline. Original pathology reports and metadata are processed through the Clinical Pathology Report Template (CPRT) with LLM assistance to extract comprehensive pathology features. These features undergo random sampling and LLM reorganization to generate CTIS-Align for model pre-training. Pathologist perform manual filtering and quality control to produce the final CTIS-Bench benchmark, ensuring clinical accuracy and diagnostic relevance.
  • Figure 3: Statistics of CTIS-Align and CTIS-Bench. The left part illustrates the word cloud of descriptions in CTIS-Align. The right part shows the question type distribution in the CTIS-Bench test set, the inner ring indicates whether the questions are open-ended or closed-ended.
  • Figure 4: Examples of CTIS-Align and CTIS-Bench datasets. CTIS-Align demonstrates a sampled and reorganized QA pair for WSI analysis. CTIS-Bench presents six key pathological aspects along with their corresponding questions
  • Figure 5: Overall Structure of the CTIS-QA Model. Left bottom part shows the prototype initialize process. The lower section illustrates the two-stage training process.