Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anatomical Positional Embeddings

Mikhail Goncharov, Valentin Samokhin, Eugenia Soboleva, Roman Sokolov, Boris Shirokikh, Mikhail Belyaev, Anvar Kurmukov, Ivan Oseledets

TL;DR

This work proposes a self-supervised model producing 3D anatomical positional embeddings of individual medical image voxels (APE) and demonstrates its superior performance compared with the existing models on anatomical landmark retrieval and weakly-supervised few-shot localization of 13 abdominal organs.

Abstract

We propose a self-supervised model producing 3D anatomical positional embeddings (APE) of individual medical image voxels. APE encodes voxels' anatomical closeness, i.e., voxels of the same organ or nearby organs always have closer positional embeddings than the voxels of more distant body parts. In contrast to the existing models of anatomical positional embeddings, our method is able to efficiently produce a map of voxel-wise embeddings for a whole volumetric input image, which makes it an optimal choice for different downstream applications. We train our APE model on 8400 publicly available CT images of abdomen and chest regions. We demonstrate its superior performance compared with the existing models on anatomical landmark retrieval and weakly-supervised few-shot localization of 13 abdominal organs. As a practical application, we show how to cheaply train APE to crop raw CT images to different anatomical regions of interest with 0.99 recall, while reducing the image volume by 10-100 times. The code and the pre-trained APE model are available at https://github.com/mishgon/ape .

Anatomical Positional Embeddings

TL;DR

This work proposes a self-supervised model producing 3D anatomical positional embeddings of individual medical image voxels (APE) and demonstrates its superior performance compared with the existing models on anatomical landmark retrieval and weakly-supervised few-shot localization of 13 abdominal organs.

Abstract

We propose a self-supervised model producing 3D anatomical positional embeddings (APE) of individual medical image voxels. APE encodes voxels' anatomical closeness, i.e., voxels of the same organ or nearby organs always have closer positional embeddings than the voxels of more distant body parts. In contrast to the existing models of anatomical positional embeddings, our method is able to efficiently produce a map of voxel-wise embeddings for a whole volumetric input image, which makes it an optimal choice for different downstream applications. We train our APE model on 8400 publicly available CT images of abdomen and chest regions. We demonstrate its superior performance compared with the existing models on anatomical landmark retrieval and weakly-supervised few-shot localization of 13 abdominal organs. As a practical application, we show how to cheaply train APE to crop raw CT images to different anatomical regions of interest with 0.99 recall, while reducing the image volume by 10-100 times. The code and the pre-trained APE model are available at https://github.com/mishgon/ape .
Paper Structure (15 sections, 2 equations, 3 figures, 4 tables)

This paper contains 15 sections, 2 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Method
  3. APE model
  4. Architecture
  5. Naïve training procedure
  6. Improved training procedure
  7. Evaluating APE on anatomical landmark retrieval
  8. Evaluating APE on few-shot organ localization
  9. Experiments & results
  10. Datasets & implementation details
  11. Anatomical landmark retrieval
  12. Few-shot organ localization
  13. Qualitative results
  14. Conclusion
  15. Disclosure of Interests.

Figures (3)

  • Figure 1: Illustration of APE training procedure. Left: sampling $n$ pairs of overlapping augmented patches and $N$ positive pairs of voxels from the overlapping regions, denoted by the markers of the same color and shape. Top-right: predicting the APE map for each patch and extracting the APE embedding for each sampled voxel. Bottom-right: the relative positions of the positive pairs of voxels in the APE embeddings' space. We train APE such that distances $d_{ij}^{\text{pred}}$ between the APE embeddings are similar to the distances $d_{ij}^{\text{true}}$ between the voxels' normalized absolute positions in the 3D physical space.
  • Figure 2: Continuous property of APE embedding maps
  • Figure 3: Projection of 3D scatter plot of APE embeddings of different organs' centers