CHAMMI: A benchmark for channel-adaptive models in microscopy imaging

TL;DR

CHAMMI tackles the problem of varying input channels in microscopy by introducing a benchmark that combines a varied-channel single-cell dataset from three public sources with a nine-task evaluation framework. The authors design and compare several channel-adaptive architectures (e.g., Depthwise, TargetParam, HyperNet) against fixed-channel baselines, showing that channel-adaptive models can generalize better to out-of-domain tasks and offer competitive computational costs. They systematically explore training regimes (random initialization vs ImageNet pretraining), data augmentations (TPS), and self-supervised signals (SimCLR), finding that hypernet-style and SSL-enhanced configurations yield robust gains on OOD tasks, especially when paired with domain-generalization methods like SWAD or MIRO. The CHAMMI benchmark, accompanied by a public dataset and evaluation API, provides a concrete path toward reusable, dataset-agnostic channel-adaptive solutions in microscopy and related imaging domains.

Abstract

Most neural networks assume that input images have a fixed number of channels (three for RGB images). However, there are many settings where the number of channels may vary, such as microscopy images where the number of channels changes depending on instruments and experimental goals. Yet, there has not been a systemic attempt to create and evaluate neural networks that are invariant to the number and type of channels. As a result, trained models remain specific to individual studies and are hardly reusable for other microscopy settings. In this paper, we present a benchmark for investigating channel-adaptive models in microscopy imaging, which consists of 1) a dataset of varied-channel single-cell images, and 2) a biologically relevant evaluation framework. In addition, we adapted several existing techniques to create channel-adaptive models and compared their performance on this benchmark to fixed-channel, baseline models. We find that channel-adaptive models can generalize better to out-of-domain tasks and can be computationally efficient. We contribute a curated dataset (https://doi.org/10.5281/zenodo.7988357) and an evaluation API (https://github.com/broadinstitute/MorphEm.git) to facilitate objective comparisons in future research and applications.

Figures (16)

• Figure 1: Example CHAMMI images (left) and illustration of a channel-adaptive model (right). The dataset consists of varying-channel images from three sources: WTC-11 hiPSC dataset (WTC-11, 3 channels), Human Protein Atlas (HPA, 4 channels), and Cell Painting datasets (CP, 5 channels). The model takes images of cells with varying number of channels and produces feature embeddings for downstream biological applications.
• Figure 2: Summary statistics of the CHAMMI dataset. A) Number of images from each source split by training and testing sets. The training set has images from all three sources, whereas the validation sets are specific to one source. B) Distribution of images with various classification labels across training and testing sets. Each image is annotated with one of six (WTC-11) or seven (HPA, CP) labels. WTC-11 images are labeled by the cell-cycle stage of single cells, HPA images are labeled by the protein subcellular localization, and the CP images are labeled by the compound treatment.
• Figure 3: Illustration of the evaluation tasks in CHAMMI with training, validation (gray) and generalization (colored) tasks. A) Cell-cycle stage classification on WTC-11 (6 classes), organized in two tasks stratified by the organelle observable in the protein channel. B) Protein localization classification on HPA (7 classes), organized in three tasks stratified by class labels and cell lines. C) Compound replicate matching on Cell Painting (7 compounds), organized in four tasks stratified by source dataset, plate ID, and compound treatment. IID: independent and identically distributed.
• Figure 4: Illustration of the evaluated models. A) Two non-adaptive, baseline approaches: ChannelReplication and FixedChannels. B) Five channel-adaptive strategies to accommodate varying image inputs: Depthwise, SliceParam, TargetParam, TemplateMixing, and HyperNet (gray blocks). Adaptive interfaces are the first layer of a shared backbone network. Descriptions are provided in Sec. \ref{['sec:architectures']} and the supplementary contains additional details.
• Figure 5: Model comparison on the CHAMMI benchmark. Radar plots have nine axes representing the benchmark tasks. Numbers in color boxes indicate the performance score obtained by the model. A) Comparison of three baseline models. The tasks in the outer circle are colored by the dataset they correspond to, with gray meaning validation tasks, and colors meaning generalization tasks. B,C) Comparison of five, channel-adaptive strategies trained or fine-tuned with the CHAMMI dataset. Number in the circle center indicates in how many tasks the model is better than the baseline.