Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Anatomy from Multiple Perspectives via Self-supervision in Chest Radiographs

Ziyu Zhou, Haozhe Luo, Mohammad Reza Hosseinzadeh Taher, Jiaxuan Pang, Xiaowei Ding, Michael B. Gotway, Jianming Liang

TL;DR

Medical imaging SSL often neglects the stability and structure of human anatomy, limiting robust representation learning. Lamps introduces three anatomy-centered perspectives—extrapolation, order coherence, and composition/decomposition—embedded in a cyclic teacher–student pretraining framework and scaled to over 1 million chest radiographs. The method yields superior transfer across segmentation, classification, and report generation, and reveals emergent properties like localizability, anatomy correspondence, and a DNA-test for part–whole hierarchy, with ablations confirming the benefits of cyclic multi-perspective training. This approach promises anatomy-aligned, generalizable features with potential to enhance explainable AI in radiology.

Abstract

Foundation models have been successful in natural language processing and computer vision because they are capable of capturing the underlying structures (foundation) of natural languages. However, in medical imaging, the key foundation lies in human anatomy, as these images directly represent the internal structures of the body, reflecting the consistency, coherence, and hierarchy of human anatomy. Yet, existing self-supervised learning (SSL) methods often overlook these perspectives, limiting their ability to effectively learn anatomical features. To overcome the limitation, we built Lamps (learning anatomy from multiple perspectives via self-supervision) pre-trained on large-scale chest radiographs by harmoniously utilizing the consistency, coherence, and hierarchy of human anatomy as the supervision signal. Extensive experiments across 10 datasets evaluated through fine-tuning and emergent property analysis demonstrate Lamps' superior robustness, transferability, and clinical potential when compared to 10 baseline models. By learning from multiple perspectives, Lamps presents a unique opportunity for foundation models to develop meaningful, robust representations that are aligned with the structure of human anatomy.

Learning Anatomy from Multiple Perspectives via Self-supervision in Chest Radiographs

TL;DR

Medical imaging SSL often neglects the stability and structure of human anatomy, limiting robust representation learning. Lamps introduces three anatomy-centered perspectives—extrapolation, order coherence, and composition/decomposition—embedded in a cyclic teacher–student pretraining framework and scaled to over 1 million chest radiographs. The method yields superior transfer across segmentation, classification, and report generation, and reveals emergent properties like localizability, anatomy correspondence, and a DNA-test for part–whole hierarchy, with ablations confirming the benefits of cyclic multi-perspective training. This approach promises anatomy-aligned, generalizable features with potential to enhance explainable AI in radiology.

Abstract

Foundation models have been successful in natural language processing and computer vision because they are capable of capturing the underlying structures (foundation) of natural languages. However, in medical imaging, the key foundation lies in human anatomy, as these images directly represent the internal structures of the body, reflecting the consistency, coherence, and hierarchy of human anatomy. Yet, existing self-supervised learning (SSL) methods often overlook these perspectives, limiting their ability to effectively learn anatomical features. To overcome the limitation, we built Lamps (learning anatomy from multiple perspectives via self-supervision) pre-trained on large-scale chest radiographs by harmoniously utilizing the consistency, coherence, and hierarchy of human anatomy as the supervision signal. Extensive experiments across 10 datasets evaluated through fine-tuning and emergent property analysis demonstrate Lamps' superior robustness, transferability, and clinical potential when compared to 10 baseline models. By learning from multiple perspectives, Lamps presents a unique opportunity for foundation models to develop meaningful, robust representations that are aligned with the structure of human anatomy.
Paper Structure (4 sections, 3 equations, 5 figures, 2 tables)

This paper contains 4 sections, 3 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Learn from Anatomy
  3. Experiments and Results
  4. Conclusion

Figures (5)

  • Figure 1: We introduce Lamps, a foundation model designed to learn from multiple perspectives. A. Lamps is built using our novel framework, which accrues knowledge from three key perspectives: extrapolation, order correction, and composition/decomposition, in a cyclic pretraining process. B. Lamps is pretrained over 1.04M images and comprehensively evaluated on 10 downstream tasks.
  • Figure 2: Lamps demonstrates anatomical understanding through emergent properties: (a) Localizability: distinguishing anatomical structures across patients; (b) Anatomy correspondence: accurately matching identical anatomies across patients (zero-shot predictions (red crosses) vs. ground truth (colored circles)).
  • Figure 3: Lamps learns hierarchical anatomies. Our model leverages DNA-test data to encode part-whole structures, enabling discrimination of whether a part structure belongs to its corresponding whole.
  • Figure 4: Lamps outperforms baselines on 7 of 8 metrics for report generation.
  • Figure 5: Lamps achieves superior generalization and robustness, surpassing SOTA SSL methods on diverse downstream tasks. For each task, statistical significance testing ($p$ < 0.05) was performed to compare Lamps with the best-performing SSL baseline.