Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Voxel Scene Graph for Intracranial Hemorrhage

Antoine P. Sanner, Nils F. Grauhan, Marc A. Brockmann, Ahmed E. Othman, Anirban Mukhopadhyay

TL;DR

This work addresses the need to model interactions between intracranial hemorrhage and adjacent brain structures by introducing a voxel-based Scene Graph Generation framework. It combines a tailored object detector with segmentation-grounded SGG variants (V-MOTIF and V-IMP) to build interpretable graphs depicting relations among Bleeding, Ventricle System, and Midline in 3D CT scans. Evaluated on INSTANCE2022 and a private cohort, the method achieves strong detection performance and recalls up to 74% of clinically relevant relations, demonstrating generalization across centers. The generated Scene Graphs provide a compact, interpretable representation that can support downstream tasks such as patient outcome prediction and treatment planning in ICH care.

Abstract

Patients with Intracranial Hemorrhage (ICH) face a potentially life-threatening condition, and patient-centered individualized treatment remains challenging due to possible clinical complications. Deep-Learning-based methods can efficiently analyze the routinely acquired head CTs to support the clinical decision-making. The majority of early work focuses on the detection and segmentation of ICH, but do not model the complex relations between ICH and adjacent brain structures. In this work, we design a tailored object detection method for ICH, which we unite with segmentation-grounded Scene Graph Generation (SGG) methods to learn a holistic representation of the clinical cerebral scene. To the best of our knowledge, this is the first application of SGG for 3D voxel images. We evaluate our method on two head-CT datasets and demonstrate that our model can recall up to 74% of clinically relevant relations. This work lays the foundation towards SGG for 3D voxel data. The generated Scene Graphs can already provide insights for the clinician, but are also valuable for all downstream tasks as a compact and interpretable representation.

Voxel Scene Graph for Intracranial Hemorrhage

TL;DR

This work addresses the need to model interactions between intracranial hemorrhage and adjacent brain structures by introducing a voxel-based Scene Graph Generation framework. It combines a tailored object detector with segmentation-grounded SGG variants (V-MOTIF and V-IMP) to build interpretable graphs depicting relations among Bleeding, Ventricle System, and Midline in 3D CT scans. Evaluated on INSTANCE2022 and a private cohort, the method achieves strong detection performance and recalls up to 74% of clinically relevant relations, demonstrating generalization across centers. The generated Scene Graphs provide a compact, interpretable representation that can support downstream tasks such as patient outcome prediction and treatment planning in ICH care.

Abstract

Patients with Intracranial Hemorrhage (ICH) face a potentially life-threatening condition, and patient-centered individualized treatment remains challenging due to possible clinical complications. Deep-Learning-based methods can efficiently analyze the routinely acquired head CTs to support the clinical decision-making. The majority of early work focuses on the detection and segmentation of ICH, but do not model the complex relations between ICH and adjacent brain structures. In this work, we design a tailored object detection method for ICH, which we unite with segmentation-grounded Scene Graph Generation (SGG) methods to learn a holistic representation of the clinical cerebral scene. To the best of our knowledge, this is the first application of SGG for 3D voxel images. We evaluate our method on two head-CT datasets and demonstrate that our model can recall up to 74% of clinically relevant relations. This work lays the foundation towards SGG for 3D voxel data. The generated Scene Graphs can already provide insights for the clinician, but are also valuable for all downstream tasks as a compact and interpretable representation.
Paper Structure (10 sections, 5 figures, 3 tables)

This paper contains 10 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experiments
  4. Results
  5. Object Detection
  6. Scene Graph Prediction
  7. Conclusion
  8. Compliance with Ethical Standards
  9. Acknowledgments.
  10. Disclosure of Interests.

Figures (5)

  • Figure 1: Example of Scene Graph for ICH. A) A slice of CT scan from an ICH patient. B) The slice with object localization. C) The associated Scene Graph.
  • Figure 2: Overview of our two-stage method for Scene Graph Generation. Objects are first detected using a hybrid detector / segmentation model. The relations are then predicted using both bounding box and segmentation mask information.
  • Figure 3: Distribution of the number of bleedings per image (upper left) and volume per bleeding (upper right) for each dataset. Distribution of the number of relations per image (bottom left) and the distribution of relations (bottom right) for each dataset.
  • Figure 4: IoU distribution of localized ventricle systems (left) and midlines (right) for each dataset. An anatomy is correctly detected if its bounding box IoU with the ground truth annotation is above 30% (red line).
  • Figure 5: Predicted segmentation and top-5 predictions of segmentation-grounded V-IMP on the test set of INSTANCE2022 (left) and from the private cohort (right). (Left) The model identifies that bleedings are flowing in the ventricle system, which may require a surgical intervention. (Right) A strong midline shift to the left is detected and attributed to the large subdural bleeding on the right side. Also note the asymmetry of the ventricle system. Both indicate an increased intracranial pressure, which may also require a different type of surgery than (left).