BLO-Inst: Bi-Level Optimization Based Alignment of YOLO and SAM for Robust Instance Segmentation

TL;DR

BLO-Inst addresses the core challenge of aligning a prompt-generating detector with a segmentation foundation model (SAM) by formulating a bi-level optimization problem over two disjoint data splits: a lower level that adapts the segmenter $oldsymbol{ extTheta}$ with fixed detector $oldsymbol{ extPhi}$ on $D_1$, and an upper level that updates the detector to minimize SAM’s validation loss on $D_2$ using the adapted segmenter. The detector is treated as a hyper-parameter, and the objective is expressed as $oldsymbol{ extPhi}^*= ext{argmin}_{oldsymbol{ extPhi}} otal(oldsymbol{ extTheta}^*(oldsymbol{ extPhi}),oldsymbol{ extPhi};D_2)$ with $oldsymbol{ extTheta}^*(oldsymbol{ extPhi})= ext{argmin}_{oldsymbol{ extTheta}} otal(oldsymbol{ extTheta},oldsymbol{ extPhi};D_1)$, enabling segmentation-aware prompting that generalizes beyond training data. BLO-Inst freezes the SAM encoder and uses LoRA to fine-tune the mask decoder, while YOLO serves as the prompt generator, resulting in a parameter-efficient system that achieves state-of-the-art or competitive performance across general and biomedical benchmarks. Extensive experiments demonstrate BLO-Inst’s superior accuracy, robustness to domain shifts, and favorable computational cost compared with standard joint-training baselines and automated-prompting methods.

Abstract

The Segment Anything Model has revolutionized image segmentation with its zero-shot capabilities, yet its reliance on manual prompts hinders fully automated deployment. While integrating object detectors as prompt generators offers a pathway to automation, existing pipelines suffer from two fundamental limitations: objective mismatch, where detectors optimized for geometric localization do not correspond to the optimal prompting context required by SAM, and alignment overfitting in standard joint training, where the detector simply memorizes specific prompt adjustments for training samples rather than learning a generalizable policy. To bridge this gap, we introduce BLO-Inst, a unified framework that aligns detection and segmentation objectives by bi-level optimization. We formulate the alignment as a nested optimization problem over disjoint data splits. In the lower level, the SAM is fine-tuned to maximize segmentation fidelity given the current detection proposals on a subset ($D_1$). In the upper level, the detector is updated to generate bounding boxes that explicitly minimize the validation loss of the fine-tuned SAM on a separate subset ($D_2$). This effectively transforms the detector into a segmentation-aware prompt generator, optimizing the bounding boxes not just for localization accuracy, but for downstream mask quality. Extensive experiments demonstrate that BLO-Inst achieves superior performance, outperforming standard baselines on tasks in general and biomedical domains.

Figures (12)

• Figure 1: Overview of the BLO-Inst framework. (a) The architecture combines a trainable YOLO detector (parameters $\Phi$) with the Segment Anything Model (SAM). (b) We employ Parameter-Efficient Fine-Tuning (PEFT) by freezing SAM's heavy encoder and injecting learnable LoRA layers (parameters $\Theta$) into the mask decoder. (c) The Bi-Level Optimization process: the lower level updates the segmenter $\Theta$ on $D_1$ to maximize mask fidelity given fixed prompts, while the upper level updates the detector $\Phi$ on $D_2$ to generate prompts that minimize the segmenter's validation loss.
• Figure 2: Illustration of the objective mismatch. Pedestrian (Left): A tighter box (Red) outperforms the precise one (Green) by reducing background clutter. Cell (Right): A larger box (Red) outperforms the precise one (Green) by providing essential context.
• Figure 3: Qualitative Results on Natural Scenes (PennFudanPed). Comparison of instance segmentation performance. Our method (far right) produces sharper masks and handles occlusions more effectively than baselines.
• Figure 4: Qualitative Results on Biomedical Imagery (CellCountIns). In this challenging dense microscopy task, BLO-Inst successfully separates tightly packed and overlapping cells, whereas other baselines tend to merge adjacent instances.
• Figure 5: Effect of Trainable Components. Comparison of fine-tuning different modules of SAM on PennFudanPed and CellCountIns datasets. Updating the mask decoder yields the optimal balance between accuracy and parameter efficiency.