VerteNet -- A Multi-Context Hybrid CNN Transformer for Accurate Vertebral Landmark Localization in Lateral Spine DXA Images

TL;DR

VerteNet tackles vertebral landmark localization in lateral DXA spine images, a domain with limited deep learning support and cross-device variability. It introduces a hybrid CNN-Transformer framework that fuses multi-context features via dual-resolution self- and cross-attention (DRSA/DRCA) and a Multi-Context Feature Fusion Block (MCFB), achieving state-of-the-art VLL on 620 DXA LSIs from multiple machines. The work also demonstrates automated abdominal aorta cropping detection and a proof-of-concept showing that IVGs derived from VerteNet landmarks can improve inter-reader agreement in AAC scoring. Together, these advances enable faster, more consistent DXA-based spinal and vascular assessments with potential clinical impact in osteoporosis and cardiovascular risk evaluation. Future work includes broader clinical validation and extending the framework to quantify kyphosis and other spinal metrics.

Abstract

Lateral Spine Image (LSI) analysis is important for medical diagnosis, treatment planning, and detailed spinal health assessments. Although modalities like Computed Tomography and Digital X-ray Imaging are commonly used, Dual Energy X-ray Absorptiometry (DXA) is often preferred due to lower radiation exposure, seamless capture, and cost-effectiveness. Accurate Vertebral Landmark Localization (VLL) on LSIs is important to detect spinal conditions like kyphosis and lordosis, as well as assessing Abdominal Aortic Calcification (AAC) using Inter-Vertebral Guides (IVGs). Nonetheless, few automated VLL methodologies have concentrated on DXA LSIs. We present VerteNet, a hybrid CNN-Transformer model featuring a novel dual-resolution attention mechanism in self and cross-attention domains, referred to as Dual Resolution Self-Attention (DRSA) and Dual Resolution Cross-Attention (DRCA). These mechanisms capture the diverse frequencies in DXA images by operating at two different feature map resolutions. Additionally, we design a Multi-Context Feature Fusion Block (MCFB) that efficiently integrates the features using DRSA and DRCA. We train VerteNet on 620 DXA LSIs from various machines and achieve superior results compared to existing methods. We also design an algorithm that utilizes VerteNet's predictions in estimating the Region of Interest (ROI) to detect potential abdominal aorta cropping, where inadequate soft tissue hinders calcification assessment. Additionally, we present a small proof-of-concept study to show that IVGs generated from VLL information can improve inter-reader correlation in AAC scoring, addressing two key areas of disagreement in expert AAC-24 scoring: IVG placement and quality control for full abdominal aorta assessment. The code for this work can be found at https://github.com/zaidilyas89/VerteNet.

Figures (7)

• Figure 1: (a) CT Lateral Spine Imaging – the gold standard, slower with highest radiation exposure ct. (b) Digital X-Ray Imaging – a faster option with lower radiation exposure than CT dxi. (c) Hologic DXA SE variant – quickest with lowest radiation exposure, though susceptible to artifacts such as bowel gas. (d) and (e) GE DXA SE and DE variants – equipped with radiation-reducing technology (black regions), offering comparatively fast imaging with low radiation exposure.
• Figure 2: (a) A DXA LSI example with red arrows marking the location of AAC. (b) An illustration of Kauppila's AAC-24 scoring method kauppila. (c) A DXA image example showing unclear vertebral boundaries, with (d) indicating the intended placement of IVGs needed for AAC-24 scoring.
• Figure 3: (a) Proposed Framework VerteNet (b) Dual Resolution Self Attention (DRSA) (c) Dual Resolution Cross Attention (DRCA) - takes two feature maps as input and generates Query from one feature map and Keys and Values from the other feature map. (d) Multi-Context Feature Fusion Block (MCFB) that employs DRSA and DRCA to calculate self-attention within, and cross attention among features from skip connection of layer n and upscaled feature map from layer n+1. Next, it utilizes a channel-wise self-attention transformer block to calculate inter-correlation among channels. (e) Conventional feature fusion approach that simply concatenates features along channels, and then performs convolution operations with activation functions (lack both inter and intra-correlation information among elements of the feature maps).
• Figure 4: (a) HiLo Attention hilosa - Uses the same window count s in both the low and actual resolution SA paths, which limits the context to individual patches. (b) DRSA - Uses the same window size in both resolution SA paths which increases the context in low-resolution SA path, and also introduces overlapping. In (a) and (b), r is the size reduction factor, and s is the window count.
• Figure 5: (a) and (b) describe the types of input images that our algorithm can process, originating from two different machines: one with black regions (GE machine) and one without (Hologic machines). (c) illustrates the block-level structure of the abdominal aorta crop detection algorithm. The image classifier categorizes the input images into two groups: those with black regions and those without (d). (e) shows the corresponding outputs for image types (a) and (b), indicating both the location and the percentage of the abdominal aorta crop detected.