Causal integration of chemical structures improves representations of microscopy images for morphological profiling

TL;DR

This work tackles the multimodal nature of high-throughput morphological profiling by proposing MICON, a representation learning framework that integrates chemical structure information during self-supervised pre-training. MICON models compounds as treatments that generate counterfactual transformations of microscopy images and optimizes a dual-contrastive objective: a Perturbation-Aware Contrastive Learning (PaCLR) term for robust image representations and a counterfactual PaCLR term that aligns synthetic counterfactuals with real perturbation representations. Across multi-source datasets and multiple evaluation regimes, MICON outperforms hand-crafted features and existing self-supervised baselines, including in settings with unseen sources and unseen compounds, and benefits further from batch-effect correction. The findings argue for explicitly modeling perturbations in morphological profiling and demonstrate the practical potential of generating and leveraging counterfactual representations to improve robustness and generalization in multimodal cellular phenotyping.

Abstract

Recent advances in self-supervised deep learning have improved our ability to quantify cellular morphological changes in high-throughput microscopy screens, a process known as morphological profiling. However, most current methods only learn from images, despite many screens being inherently multimodal, as they involve both a chemical or genetic perturbation as well as an image-based readout. We hypothesized that incorporating chemical compound structure during self-supervised pre-training could improve learned representations of images in high-throughput microscopy screens. We introduce a representation learning framework, MICON (Molecular-Image Contrastive Learning), that models chemical compounds as treatments that induce counterfactual transformations of cell phenotypes. MICON significantly outperforms classical hand-crafted features such as CellProfiler and existing deep-learning-based representation learning methods in challenging evaluation settings where models must identify reproducible effects of drugs across independent replicates and data-generating centers. We demonstrate that incorporating chemical compound information into the learning process provides consistent improvements in our evaluation setting and that modeling compounds specifically as treatments in a causal framework outperforms approaches that directly align images and compounds in a single representation space. Our findings point to a new direction for representation learning in morphological profiling, suggesting that methods should explicitly account for the multimodal nature of microscopy screening data.

Figures (9)

• Figure 1: Summary of the MICON framework.(A) An overview of our model. Compound perturbed (blue) images and negative control (orange) images are transformed into representations by an image encoder, while chemical compounds (purple) are transformed into representations by a compound encoder. A fusion module accepts pairs of negative control representations and compound representations and outputs counterfactual representations (light blue). All representations are passed through a projection layer to form the final projected representations. (B) Summary of the pertubation-aware contrastive (PAC) loss. Perturbed images (blue) and negative controls (orange) from the same batches as the perturbed images (batches denoted by small colored boxes) are sampled, with at least two images for each perturbation. Images treated with the same perturbation (a, b, or c) serve as positive pairs (black squares) for the PaCLR loss. (C) Summary of the counterfactual PaCLR loss. Perturbation representations (blue) and generated counterfactual representations (light blue) using the same compound (a, b, or c) serve as positive pairs (black squares) for the counterfactual PaCLR loss.
• Figure 2: Causal diagram of apparent phenotype in image-based perturbation profiling. In addition to a perturbation treatment's effect on cells and, in turn, apparent phenotypes, microscopy batch effects can impart multiple layers of impact: at the levels of perturbation treatment, cell phenotype, and apparent imaging phenotype.
• Figure 3: Stratification of positive control wells into in-distribution and out-of-distribution datasets.(A) POS-CTL wells from a total of six sources are used. For our out-of-distribution evaluation, five sources are designated as seen (blue boxes) during model training, and 1 source is designated as unseen (orange boxes). In panels B and C, filled parts of the boxes indicate data from each source in that split, while unfilled (white) parts of the boxes indicate held-out data (note that filled/unfilled ratios are not proportionately scaled to split ratios, and are just intended to illustrate where splits are disjoint). (B) The in-distribution evaluation trains the representation learning model on a subset of microscopy batches from all six sources, holding out one batch from each source for validation. The training and validation dataset are used as the retrieval dataset for compound-replicate matching, with unseen microscopy batches from all six sources used as the query dataset. (C) The out-of-distribution evaluation trains and validates the representation learning model on the five seen sources. For evaluation, the retrieval dataset is the training and validation data plus a subset of microscopy batches from the unseen source, and the query dataset is a disjoint subset of microscopy batches from the unseen source.
• Figure 4: Stratification of Target-2 and Compound plates into in-distribution and out-of-distribution datasets.(A) The 301 Target-2 compounds (blue) are designated as seen compounds for training representation learning models, while 184 unseen compounds (orange) are sampled from the Compound plates and held out. (B) The in-distribution evaluation trains the representation learning model on 80% of microscopy batches from all Target-2 plates, reserving 20% of microscopy batches for the query dataset. (C) The out-of-distribution evaluation trains the representation learning model on all microscopy batches from the Target-2 plates. The training dataset is combined with 493 wells for 184 unseen compounds to form the retrieval dataset and evaluated on a query dataset of a disjoint set of 368 wells of held-out, unseen compounds from the Compound plates.
• Figure 5: Compound-replicate retrieval accuracy for representation learning methods across evaluation settings. Retrieval accuracies on the (A) POS-CTL ID query dataset, (B) Target-2 ID query dataset, (C) POS-CTL OOD query dataset (unseen source), and (D) Target-2 OOD query dataset (unseen compounds). NSB designates retrieval of the nearest neighbor not in the same batch; NSS designates retrieval of the nearest neighbor not in the same source. $n$=3 models trained and evaluated with different dataset stratifications and different random seeds; the mean accuracy is labeled, and the error bars represent standard deviation. Asterisk indicates that MICON significantly outperforms the baseline, defined as $p<0.05$ on an unpaired one-tailed t-test with performance across the $n$=3 random seeds as trials.