X-WIN: Building Chest Radiograph World Model via Predictive Sensing

TL;DR

X-WIN addresses the gap of 3D spatial understanding in chest radiographs by distilling volumetric knowledge from CT into a CXR-capable world model. It learns to predict CT-derived 2D projections under 3D transformations using an action-conditioned framework, and optimizes representations with affinity-guided contrastive alignment, masked image modeling, and structure-preserving domain adaptation. The approach yields state-of-the-art linear-probe performance and strong few-shot results on COVID-19 detection, while also enabling 3D CT reconstruction from radiographic views. This work demonstrates the feasibility and value of cross-domain 3D knowledge transfer for radiographic analysis, with potential impact on accuracy, generalization, and interpretability in clinical workflows.

Abstract

Chest X-ray radiography (CXR) is an essential medical imaging technique for disease diagnosis. However, as 2D projectional images, CXRs are limited by structural superposition and hence fail to capture 3D anatomies. This limitation makes representation learning and disease diagnosis challenging. To address this challenge, we propose a novel CXR world model named X-WIN, which distills volumetric knowledge from chest computed tomography (CT) by learning to predict its 2D projections in latent space. The core idea is that a world model with internalized knowledge of 3D anatomical structure can predict CXRs under various transformations in 3D space. During projection prediction, we introduce an affinity-guided contrastive alignment loss that leverages mutual similarities to capture rich, correlated information across projections from the same volume. To improve model adaptability, we incorporate real CXRs into training through masked image modeling and employ a domain classifier to encourage statistically similar representations for real and simulated CXRs. Comprehensive experiments show that X-WIN outperforms existing foundation models on diverse downstream tasks using linear probing and few-shot fine-tuning. X-WIN also demonstrates the ability to render 2D projections for reconstructing a 3D CT volume.

Figures (6)

• Figure 1: Illustration of our motivation. (a) Given routine frontal and lateral CXRs, radiologists can reconstruct a 3D model of the chest volume based on their anatomical knowledge. This capability helps them make diagnostic decisions despite the occlusion of structures. (b) Inspired by our observation in (a), we propose a novel chest X-ray world intelligence network (X-WIN) that builds a world model by predicting new projection views under various transformations of an X-ray source.
• Figure 2: Overview of the X-WIN framework. Top: Our proposed approach distills spatial knowledge from CT by learning to predict its projections in latent space. We generate new projections by rotating a X-ray source controlled by actions. Bottom: Masked image modeling and a domain classifier are incorporated to learn a cohesive representation space between real and simulated domains. EMA denotes exponential moving average.
• Figure 3: t-SNE visualizations of encoder representations with corresponding semantic labels. The representations of X-WIN demonstrates a modest separation between semantic classes without finetuning. With 4-shot fine-tuning, clear clusters for three classes can be observed.
• Figure 4: Hyperparameter tuning. Left: Effects of loss functions (circles) and sensitivity analysis on loss weights (lines). Right: Analysis of the impact of rotation step sizes on model performance. We reported performance on VinDr via linear probing.
• Figure 5: Analysis of representation correspondences. We compute cosine similarities between the anatomical landmarks in the simulated CXRs and the representations of the real CXRs. Regions highlighted in yellow indicate patch representations with high similarities.