Structurally Different Neural Network Blocks for the Segmentation of Atrial and Aortic Perivascular Adipose Tissue in Multi-centre CT Angiography Scans

TL;DR

This study introduces LegoNet, a neural network that alternates structurally distinct encoder blocks (SE, Swin, UX) to improve 3D segmentation of the IMA, aorta, and PVAT from multi-centre CTA scans. It presents three LegoNet configurations and demonstrates superior performance over leading CNN and ViT models on in-house data, with extensive external validation including a large US cohort and public ASOCA data, aided by iterative refinement with clinician corrections. The approach achieves high Dice Similarity Coefficients and strong observer-level agreement, underscoring its potential for prognostic radiomics and personalized cardiovascular risk assessment. By enabling robust, automated vascular and perivascular segmentation across diverse datasets, LegoNet supports scalable clinical decision support and cardiovascular management.

Abstract

Since the emergence of convolutional neural networks (CNNs) and, later, vision transformers (ViTs), deep learning architectures have predominantly relied on identical block types with varying hyperparameters. We propose a novel block alternation strategy to leverage the complementary strengths of different architectural designs, assembling structurally distinct components similar to Lego blocks. We introduce LegoNet, a deep learning framework that alternates CNN-based and SwinViT-based blocks to enhance feature learning for medical image segmentation. We investigate three variations of LegoNet and apply this concept to a previously unexplored clinical problem: the segmentation of the internal mammary artery (IMA), aorta, and perivascular adipose tissue (PVAT) from computed tomography angiography (CTA) scans. These PVAT regions have been shown to possess prognostic value in assessing cardiovascular risk and primary clinical outcomes. We evaluate LegoNet on large datasets, achieving superior performance to other leading architectures. Furthermore, we assess the model's generalizability on external testing cohorts, where an expert clinician corrects the model's segmentations, achieving DSC > 0.90 across various external, international, and public cohorts. To further validate the model's clinical reliability, we perform intra- and inter-observer variability analysis, demonstrating strong agreement with human annotations. The proposed methodology has significant implications for diagnostic cardiovascular management and early prognosis, offering a robust, automated solution for vascular and perivascular segmentation and risk assessment in clinical practice, paving the way for personalised medicine.

Figures (4)

• Figure 1: The figure shows the inner structure of each block type used for our model construction. (a) is the squeeze-and-excitation block; (b) is the Swin block; and (c) is the UX block.
• Figure 2: The figure shows the LegoNet (specifically, LegoNet-2) architecture. $F_{1-4}$ indicate the feature size, which is set to $\{24,48,96,192\}$, and $S$ is the hidden size, set to 768. This typical U-shaped architecture utilizes the block alternation concept, switching between Swin and SE blocks in the encoder in this example. The decoder is kept the same for all the variations of the model.
• Figure 3: Model Refinement via Iterative Learning. This approach improves the segmentation model to maturity before deploying it in large cohort data. The model is initially trained with a small, feasible cohort and is internally validated. Several cohorts of data are then used to improve the model by adding more value to the learning process iteratively.
• Figure 4: The figure shows the correlation and Bland-Altman plots for three external cohorts, comparing the model's prediction and clinician's segmentation masks.