Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiCLIP: Bidirectional and Consistent Language-Image Processing for Robust Medical Image Segmentation

Saivan Talaei, Fatemeh Daneshfar, Abdulhady Abas Abdullah, Mustaqeem Khan

TL;DR

This work introduces BiCLIP (Bidirectional and Consistent Language-Image Processing), a framework engineered to bolster robustness in medical segmentation and implements an augmentation consistency objective that regularizes intermediate representations against perturbed input views.

Abstract

Medical image segmentation is a cornerstone of computer-assisted diagnosis and treatment planning. While recent multimodal vision-language models have shown promise in enhancing semantic understanding through textual descriptions, their resilience in "in-the-wild" clinical settings-characterized by scarce annotations and hardware-induced image degradations-remains under-explored. We introduce BiCLIP (Bidirectional and Consistent Language-Image Processing), a framework engineered to bolster robustness in medical segmentation. BiCLIP features a bidirectional multimodal fusion mechanism that enables visual features to iteratively refine textual representations, ensuring superior semantic alignment. To further stabilize learning, we implement an augmentation consistency objective that regularizes intermediate representations against perturbed input views. Evaluation on the QaTa-COV19 and MosMedData+ benchmarks demonstrates that BiCLIP consistently surpasses state-of-the-art image-only and multimodal baselines. Notably, BiCLIP maintains high performance when trained on as little as 1% of labeled data and exhibits significant resistance to clinical artifacts, including motion blur and low-dose CT noise.

BiCLIP: Bidirectional and Consistent Language-Image Processing for Robust Medical Image Segmentation

TL;DR

This work introduces BiCLIP (Bidirectional and Consistent Language-Image Processing), a framework engineered to bolster robustness in medical segmentation and implements an augmentation consistency objective that regularizes intermediate representations against perturbed input views.

Abstract

Medical image segmentation is a cornerstone of computer-assisted diagnosis and treatment planning. While recent multimodal vision-language models have shown promise in enhancing semantic understanding through textual descriptions, their resilience in "in-the-wild" clinical settings-characterized by scarce annotations and hardware-induced image degradations-remains under-explored. We introduce BiCLIP (Bidirectional and Consistent Language-Image Processing), a framework engineered to bolster robustness in medical segmentation. BiCLIP features a bidirectional multimodal fusion mechanism that enables visual features to iteratively refine textual representations, ensuring superior semantic alignment. To further stabilize learning, we implement an augmentation consistency objective that regularizes intermediate representations against perturbed input views. Evaluation on the QaTa-COV19 and MosMedData+ benchmarks demonstrates that BiCLIP consistently surpasses state-of-the-art image-only and multimodal baselines. Notably, BiCLIP maintains high performance when trained on as little as 1% of labeled data and exhibits significant resistance to clinical artifacts, including motion blur and low-dose CT noise.
Paper Structure (12 sections, 8 equations, 2 figures, 3 tables)

This paper contains 12 sections, 8 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Bidirectional Multimodal Fusion Module
  4. Image Augmentation Consistency Module
  5. Experiments and Results
  6. Settings
  7. Comparison with State-Of-The-Art Methods
  8. Robustness Under Low-Data Regimes
  9. Robustness to Noisy Test Data
  10. Conclusion

Figures (2)

  • Figure 1: Overview of the BiCLIP framework. (a) Overall multimodal segmentation architecture; (b) pseudo-image generator for bidirectional vision-language interaction; (c) IAC module for robust learning.
  • Figure 2: Qualitative comparison of predictions on clean test images and on test images degraded by low-dose CT noise (DL-140) and motion blur (MB-K7; kernel size 7).