Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Radiology and Pathology: A DICOM-Based Framework for Multimodal Mapping and Integrated Visualization

Nilesh P. Rijhwani, Titus J. Brinker, Peter Neher, Marco Nolden, Klaus Maier-Hein, Maximilian Fischer, Christoph Wies

TL;DR

The paper tackles the challenge of integrating radiology and pathology multimodal data across disparate systems by presenting a DICOM-centered, modular framework and a web-based Combined Viewer that synchronizes radiology and histopathology data. It introduces a end-to-end pipeline including DICOM-WSI conversion, anatomical localization, and automated segmentation to produce DICOM-SEG objects, all linked to the original radiology data and WSIs. The approach leverages existing tools (dcm2niix, TotalSegmentator, dcmqi) and open-source viewers (OHIF, SLIM) within the Kaapana ecosystem to enable reproducible, cross-modal exploration and visualization. This framework supports scalable multimodal analysis, facilitates cross-disciplinary collaboration, and offers a foundation for future clinical integration and validation of radiology-pathology diagnostics.

Abstract

Accurate disease diagnosis depends on effective collaboration between medical specialties, yet departments often use distinct data systems and proprietary formats. This heterogeneity hinders joint analysis and integration of complementary diagnostic information. The use of separate viewers for each modality further restricts cross-specialty collaboration. Although multimodal integration, particularly between radiology and pathology, has demonstrated potential for identifying novel biomarkers, it still relies heavily on manual, time-consuming data pairing. This project introduces an interdisciplinary toolbox that can operate within the Kaapana framework or as a standalone tool to bridge radiology and pathology. By linking modalityspecific viewers and extending them with automated image registration and alignment, the platform enables efficient, scalable multimodal analysis. The integrated environment promotes reproducible workflows, accelerates crossdisciplinary research, and facilitates deeper insights into disease mechanisms and patient care.

Bridging Radiology and Pathology: A DICOM-Based Framework for Multimodal Mapping and Integrated Visualization

TL;DR

The paper tackles the challenge of integrating radiology and pathology multimodal data across disparate systems by presenting a DICOM-centered, modular framework and a web-based Combined Viewer that synchronizes radiology and histopathology data. It introduces a end-to-end pipeline including DICOM-WSI conversion, anatomical localization, and automated segmentation to produce DICOM-SEG objects, all linked to the original radiology data and WSIs. The approach leverages existing tools (dcm2niix, TotalSegmentator, dcmqi) and open-source viewers (OHIF, SLIM) within the Kaapana ecosystem to enable reproducible, cross-modal exploration and visualization. This framework supports scalable multimodal analysis, facilitates cross-disciplinary collaboration, and offers a foundation for future clinical integration and validation of radiology-pathology diagnostics.

Abstract

Accurate disease diagnosis depends on effective collaboration between medical specialties, yet departments often use distinct data systems and proprietary formats. This heterogeneity hinders joint analysis and integration of complementary diagnostic information. The use of separate viewers for each modality further restricts cross-specialty collaboration. Although multimodal integration, particularly between radiology and pathology, has demonstrated potential for identifying novel biomarkers, it still relies heavily on manual, time-consuming data pairing. This project introduces an interdisciplinary toolbox that can operate within the Kaapana framework or as a standalone tool to bridge radiology and pathology. By linking modalityspecific viewers and extending them with automated image registration and alignment, the platform enables efficient, scalable multimodal analysis. The integrated environment promotes reproducible workflows, accelerates crossdisciplinary research, and facilitates deeper insights into disease mechanisms and patient care.
Paper Structure (11 sections, 2 figures)

This paper contains 11 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. Data preprocessing
  4. Dicom conversion
  5. Localization of WSI in radiology
  6. Segmentation pipeline
  7. Combined Viewer
  8. Human Interaction
  9. Implementation
  10. Results
  11. Discussion

Figures (2)

  • Figure 1.1: End-to-end pipeline. The flow runs from the user action in the web UI to a DICOM SEG created by dcmqi, with SM-driven body-part inference and master JSON mapping in the middle.
  • Figure 1.2: The screenshot of the combined split-viewer which demonstrates simultaneous viewing of images with auto-segmented "Prostate" along with clickable overlay box