CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detection

TL;DR

CheXLearner addresses the challenge of fine-grained, region-wise progression detection in temporal chest X-rays by unifying DETR-based anatomical region localization with a Med-MAM hyperbolic manifold alignment and text-guided supervision from MedCLIP. The framework jointly optimizes detection, cross-modal alignment, and progression classification in an end-to-end manner, using Euclidean fusion and Riemannian parallel transport to reduce domain shifts and preserve semantic discrepancies. Empirical results show substantial gains in region-level progression accuracy (Acc $=81.12\%$, F1 $=80.32\%$) and downstream disease AUC ($=91.52\%$), with notable improvements in structurally complex regions. This approach demonstrates the value of integrating anatomical structure alignment with region-wise textual semantics for robust medical image analysis and has potential to enhance clinical decision support and resource planning.

Abstract

Temporal medical image analysis is essential for clinical decision-making, yet existing methods either align images and text at a coarse level - causing potential semantic mismatches - or depend solely on visual information, lacking medical semantic integration. We present CheXLearner, the first end-to-end framework that unifies anatomical region detection, Riemannian manifold-based structure alignment, and fine-grained regional semantic guidance. Our proposed Med-Manifold Alignment Module (Med-MAM) leverages hyperbolic geometry to robustly align anatomical structures and capture pathologically meaningful discrepancies across temporal chest X-rays. By introducing regional progression descriptions as supervision, CheXLearner achieves enhanced cross-modal representation learning and supports dynamic low-level feature optimization. Experiments show that CheXLearner achieves 81.12% (+17.2%) average accuracy and 80.32% (+11.05%) F1-score on anatomical region progression detection - substantially outperforming state-of-the-art baselines, especially in structurally complex regions. Additionally, our model attains a 91.52% average AUC score in downstream disease classification, validating its superior feature representation.

Figures (5)

• Figure 1: (a) Methods (e.g., MLRG liu2025enhanced, MLIP li2024mlip) that rely on coarse image-report alignment. (b) Methods (e.g., CheXDetector) eshraghi2024representation that detect anatomical region progression but ignore semantic integration and suffer from regional noise. (c) Our proposed end-to-end model with anatomical structure alignment and regional progression semantics integration.
• Figure 2: Overview of the CheXLearner framework. The model extracts region-of-interest (ROI) features from temporal CXRs using a DETR pretrained on CXR data. To address anatomical structure misalignment, the Med-MAM module mitigates domain shifts on Manifold and generates discrepancy features. Feature learning is facilitated through fine-grained image-text contrastive (ITC) and image-text matching (ITM) tasks that jointly model anatomical regions and disease progression description, with end-to-end optimization of the visual low-level features during training.
• Figure 3: Euclidean Dynamic Context-Difference Fusion module.
• Figure 4: Manifold difference modeling.
• Figure 5: Scatter Plot of Feature Distributions for Our CheXLearner (upper region) vs. CheXDetector (lower region).