ViM-UNet: Vision Mamba for Biomedical Segmentation
Anwai Archit, Constantin Pape
TL;DR
Biomedical segmentation with varying context requirements benefits from global receptive fields, but transformers are computationally heavy. ViM-UNet leverages Vision Mamba to provide a global field of view with higher efficiency than transformers, combining a ViM encoder with a UNet-like decoder. In experiments on LIVECell and CREMI, ViM-UNet matches or surpasses UNet and consistently outperforms UNETR, while using fewer parameters and memory. The work positions ViM-UNet as a practical alternative for large-context biomedical tasks and highlights potential extensions to 3D segmentation and tracking, with open-source code available.
Abstract
CNNs, most notably the UNet, are the default architecture for biomedical segmentation. Transformer-based approaches, such as UNETR, have been proposed to replace them, benefiting from a global field of view, but suffering from larger runtimes and higher parameter counts. The recent Vision Mamba architecture offers a compelling alternative to transformers, also providing a global field of view, but at higher efficiency. Here, we introduce ViM-UNet, a novel segmentation architecture based on it and compare it to UNet and UNETR for two challenging microscopy instance segmentation tasks. We find that it performs similarly or better than UNet, depending on the task, and outperforms UNETR while being more efficient. Our code is open source and documented at https://github.com/constantinpape/torch-em/blob/main/vimunet.md.