Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differential Attention-Augmented BiomedCLIP with Asymmetric Focal Optimization for Imbalanced Multi-Label Video Capsule Endoscopy Classification

Podakanti Satyajith Chary, Nagarajan Ganapathy

Abstract

This work presents a multi-label classification framework for video capsule endoscopy (VCE) that addresses the extreme class imbalance inherent in the Galar dataset through a combination of architectural and optimization-level strategies. Our approach modifies BiomedCLIP, a biomedical vision-language foundation model, by replacing its standard multi-head self-attention with a differential attention mechanism that computes the difference between two softmax attention maps to suppress attention noise. To counteract the skewed label distribution, where pathological findings constitute less than 0.1% of all annotated frames, a sqrt-frequency weighted sampler, asymmetric focal loss, mixup regularization, and per-class threshold optimization are employed. Temporal coherence is enforced through median-filter smoothing and gap merging prior to event-level JSON generation. On the held-out RARE-VISION test set comprising three NaviCam examinations (161,025 frames), the pipeline achieves an overall temporal mAP@0.5 of 0.2456 and mAP@0.95 of 0.2353, with total inference completed in approximately 8.6 minutes on a single GPU.

Differential Attention-Augmented BiomedCLIP with Asymmetric Focal Optimization for Imbalanced Multi-Label Video Capsule Endoscopy Classification

Abstract

This work presents a multi-label classification framework for video capsule endoscopy (VCE) that addresses the extreme class imbalance inherent in the Galar dataset through a combination of architectural and optimization-level strategies. Our approach modifies BiomedCLIP, a biomedical vision-language foundation model, by replacing its standard multi-head self-attention with a differential attention mechanism that computes the difference between two softmax attention maps to suppress attention noise. To counteract the skewed label distribution, where pathological findings constitute less than 0.1% of all annotated frames, a sqrt-frequency weighted sampler, asymmetric focal loss, mixup regularization, and per-class threshold optimization are employed. Temporal coherence is enforced through median-filter smoothing and gap merging prior to event-level JSON generation. On the held-out RARE-VISION test set comprising three NaviCam examinations (161,025 frames), the pipeline achieves an overall temporal mAP@0.5 of 0.2456 and mAP@0.95 of 0.2353, with total inference completed in approximately 8.6 minutes on a single GPU.
Paper Structure (16 sections, 4 equations, 1 figure, 2 tables)

This paper contains 16 sections, 4 equations, 1 figure, 2 tables.

Table of Contents

  1. Motivation
  2. Methods
  3. Backbone: Differential BiomedCLIP
  4. Classification Architecture
  5. How was class imbalance handled?
  6. Regularization
  7. Optimization
  8. Temporal Post-Processing
  9. Results
  10. Competition Metrics
  11. Validation Performance
  12. Training Dynamics
  13. Test Inference
  14. Discussion
  15. Summary
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Overview of the proposed Differential BiomedCLIP pipeline. Stage 1: VCE frames are processed through a sqrt-frequency sampler and augmented with mixup ($\alpha=0.3$) and geometric/photometric transforms. Stage 2: The differential ViT backbone (Patch16, 12 blocks, 768-d) extracts 512-d image features, which are routed to both an excitation-gated classification head (17 classes) and a contrastive alignment module against PubMedBERT-encoded clinical text prompts. EMA ($\beta=0.999$) maintains a smoothed parameter copy. Stage 3: Asymmetric focal loss and contrastive BCE loss are combined as $\mathcal{L}_{\text{cls}} + 0.4\mathcal{L}_{\text{con}}$. Stage 4: Per-class optimized thresholds, temporal median smoothing ($w=7$), and gap merging ($\leq 5$ frames) convert frame-level posteriors into the competition event JSON.