ZACH-ViT: A Zero-Token Vision Transformer with ShuffleStrides Data Augmentation for Robust Lung Ultrasound Classification

TL;DR

ZACH-ViT introduces a zero-token, permutation-invariant Vision Transformer designed for small, heterogeneous lung ultrasound datasets. By removing positional embeddings and CLS tokens and coupling with ShuffleStrides Data Augmentation, the model achieves robust, balanced discrimination between cardiogenic oedema and a heterogeneous non-cardiogenic class, with ROC-AUCs of 0.80 (validation) and 0.79 (test) using only 0.25M parameters. The approach demonstrates that architectural simplicity aligned with data structure, plus clinically informed augmentation, can surpass larger models on limited data and enable real-time deployment. The work provides open-source code and a reproducible pipeline, and suggests broad applicability to unordered medical image data and other domains with weak spatial ordering.

Abstract

Differentiating cardiogenic pulmonary oedema (CPE) from non-cardiogenic and structurally normal lungs in lung ultrasound (LUS) videos remains challenging due to the high visual variability of non-cardiogenic inflammatory patterns (NCIP/ARDS-like), interstitial lung disease, and healthy lungs. This heterogeneity complicates automated classification as overlapping B-lines and pleural artefacts are common. We introduce ZACH-ViT (Zero-token Adaptive Compact Hierarchical Vision Transformer), a 0.25 M-parameter Vision Transformer variant that removes both positional embeddings and the [CLS] token, making it fully permutation-invariant and suitable for unordered medical image data. To enhance generalization, we propose ShuffleStrides Data Augmentation (SSDA), which permutes probe-view sequences and frame orders while preserving anatomical validity. ZACH-ViT was evaluated on 380 LUS videos from 95 critically ill patients against nine state-of-the-art baselines. Despite the heterogeneity of the non-cardiogenic group, ZACH-ViT achieved the highest validation and test ROC-AUC (0.80 and 0.79) with balanced sensitivity (0.60) and specificity (0.91), while all competing models collapsed to trivial classification. It trains 1.35x faster than Minimal ViT (0.62M parameters) with 2.5x fewer parameters, supporting real-time clinical deployment. These results show that aligning architectural design with data structure can outperform scale in small-data medical imaging.

Figures (4)

• Figure 1: Region of interest (ROI) extraction pipeline. (Left) Original LUS frame; (Middle) intensity thresholding to suppress background; (Right) pleural-line-centered crop retaining B-lines and pleural artifacts. This standardized ROI ensures focus on diagnostically relevant structures across patients.
• Figure 2: Height Reduction to 50% and final ROI.
• Figure 3: Video-to-image transformation modes. VI: horizontal frame concatenation; SVI: shuffled frames; VIS: vertical stacking of four anatomical views. ShuffleStrides Data Augmentation (SSDA) permutes the order of these views (e.g., [1,2,3,4] → [1,2,4,3]) while preserving anatomical validity.
• Figure 4: Training and validation ROC-AUC per epoch across all datasets and models. Per-epoch training (left) and validation (right) ROC-AUC curves are shown for all augmentation regimes. Each row corresponds to a dataset (top: VIS, middle: 0-SSDA, bottom: 0_2-SSDA). ZACH-ViT (blue dashed) consistently achieves the highest ROC-AUC and demonstrates the most stable convergence across all regimes, particularly under semi-supervised augmentation. Here standard ViT refers to the Minimal ViT.