Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anatomy-guided Pathology Segmentation

Alexander Jaus, Constantin Seibold, Simon Reiß, Lukas Heine, Anton Schily, Moon Kim, Fin Hendrik Bahnsen, Ken Herrmann, Rainer Stiefelhagen, Jens Kleesiek

TL;DR

The paper addresses the limited generalization of pathology segmentation models that ignore anatomical context by introducing APEx, a query-based, anatomy-pathology joint segmentation framework. APEx uses a shared embedding space, two decoders for anatomy and pathology, and an anatomy-to-pathology communication module to make pathology predictions informed by the patient’s anatomy. Through extensive ablations and two diverse datasets (FDG-PET-CT and Chest X-Ray), APEx consistently improves over strong baselines, achieving up to around a 3.3% gain in segmentation metrics and notable gains in mAP for instance segmentation. The approach reflects a clinically plausible workflow by embedding anatomical knowledge directly into pathology predictions and is positioned to enhance reliability and interpretability of automated pathology segmentation, with code to be released publicly.

Abstract

Pathological structures in medical images are typically deviations from the expected anatomy of a patient. While clinicians consider this interplay between anatomy and pathology, recent deep learning algorithms specialize in recognizing either one of the two, rarely considering the patient's body from such a joint perspective. In this paper, we develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features. Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy and interleaves them via a mixing strategy into the pathology-decoder for anatomy-informed pathology predictions. In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods. Code and models will be publicly available at github.com/alexanderjaus/APEx.

Anatomy-guided Pathology Segmentation

TL;DR

The paper addresses the limited generalization of pathology segmentation models that ignore anatomical context by introducing APEx, a query-based, anatomy-pathology joint segmentation framework. APEx uses a shared embedding space, two decoders for anatomy and pathology, and an anatomy-to-pathology communication module to make pathology predictions informed by the patient’s anatomy. Through extensive ablations and two diverse datasets (FDG-PET-CT and Chest X-Ray), APEx consistently improves over strong baselines, achieving up to around a 3.3% gain in segmentation metrics and notable gains in mAP for instance segmentation. The approach reflects a clinically plausible workflow by embedding anatomical knowledge directly into pathology predictions and is positioned to enhance reliability and interpretability of automated pathology segmentation, with code to be released publicly.

Abstract

Pathological structures in medical images are typically deviations from the expected anatomy of a patient. While clinicians consider this interplay between anatomy and pathology, recent deep learning algorithms specialize in recognizing either one of the two, rarely considering the patient's body from such a joint perspective. In this paper, we develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features. Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy and interleaves them via a mixing strategy into the pathology-decoder for anatomy-informed pathology predictions. In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods. Code and models will be publicly available at github.com/alexanderjaus/APEx.
Paper Structure (14 sections, 4 equations, 2 figures, 3 tables)

This paper contains 14 sections, 4 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work:
  3. Methodology
  4. Preliminaries
  5. Incorporating Learned Anatomical Knowlege: A roadmap
  6. Architecture Ablations:
  7. Proposed Approach: APEx
  8. Shared Embedding Architecture:
  9. Anatomy and Pathology Decoders:
  10. Anatomy to Pathology Communication Strategy
  11. Joint Anatomy and Pathology Segmentation
  12. Experiments and Results
  13. Semantic- and Instance Segmentation Results
  14. Conclusion

Figures (2)

  • Figure 1: Overview of the proposed APEx Method, leveraging a shared pixel encoder, shared pixel embedding space, separate decoders and a query-mixing module.
  • Figure 2: Stacked 2D tumor predictions next to top-5 attended anatomical structures.