Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual Advancement of Representation Learning and Clustering for Sparse and Noisy Images

Wenlin Li, Yucheng Xu, Xiaoqing Zheng, Suoya Han, Jun Wang, Xiaobo Sun

TL;DR

DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions, and utilizes a Student's t mixture model to achieve more robust and adaptable clustering of SNIs.

Abstract

Sparse and noisy images (SNIs), like those in spatial gene expression data, pose significant challenges for effective representation learning and clustering, which are essential for thorough data analysis and interpretation. In response to these challenges, we propose Dual Advancement of Representation Learning and Clustering (DARLC), an innovative framework that leverages contrastive learning to enhance the representations derived from masked image modeling. Simultaneously, DARLC integrates cluster assignments in a cohesive, end-to-end approach. This integrated clustering strategy addresses the "class collision problem" inherent in contrastive learning, thus improving the quality of the resulting representations. To generate more plausible positive views for contrastive learning, we employ a graph attention network-based technique that produces denoised images as augmented data. As such, our framework offers a comprehensive approach that improves the learning of representations by enhancing their local perceptibility, distinctiveness, and the understanding of relational semantics. Furthermore, we utilize a Student's t mixture model to achieve more robust and adaptable clustering of SNIs. Extensive experiments, conducted across 12 different types of datasets consisting of SNIs, demonstrate that DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions. Code is available at https://github.com/zipging/DARLC.

Dual Advancement of Representation Learning and Clustering for Sparse and Noisy Images

TL;DR

DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions, and utilizes a Student's t mixture model to achieve more robust and adaptable clustering of SNIs.

Abstract

Sparse and noisy images (SNIs), like those in spatial gene expression data, pose significant challenges for effective representation learning and clustering, which are essential for thorough data analysis and interpretation. In response to these challenges, we propose Dual Advancement of Representation Learning and Clustering (DARLC), an innovative framework that leverages contrastive learning to enhance the representations derived from masked image modeling. Simultaneously, DARLC integrates cluster assignments in a cohesive, end-to-end approach. This integrated clustering strategy addresses the "class collision problem" inherent in contrastive learning, thus improving the quality of the resulting representations. To generate more plausible positive views for contrastive learning, we employ a graph attention network-based technique that produces denoised images as augmented data. As such, our framework offers a comprehensive approach that improves the learning of representations by enhancing their local perceptibility, distinctiveness, and the understanding of relational semantics. Furthermore, we utilize a Student's t mixture model to achieve more robust and adaptable clustering of SNIs. Extensive experiments, conducted across 12 different types of datasets consisting of SNIs, demonstrate that DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions. Code is available at https://github.com/zipging/DARLC.
Paper Structure (26 sections, 26 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 26 sections, 26 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Self-supervised Representation Learning for Images
  4. Deep Image Clustering.
  5. Methodology
  6. Overview
  7. Unified Self-supervised Representation Learning (Module I)
  8. Denoising-based Data Augmentation.
  9. Unified Self-supervised Image Representation Learning.
  10. Self-paced Deep Image Clustering (Module II)
  11. Student’s t mixture model.
  12. Self-paced Joint Optimization of Image Embeddings and Cluster Assignments.
  13. Experiments
  14. Experimental Settings
  15. Datasets.
  16. ...and 11 more sections

Figures (3)

  • Figure 1: An example of sparse and noisy SGEP image is displayed in the second panel to the left. The gene expression levels across space are represented by pixel brightness, while void expression areas are displayed in purple. The first panel showcases the regions (cortical layer V) where the gene is significantly expressed, namely the major gene expression pattern. The right two panels illustrate the tasks and applications that utilize SGEP images.
  • Figure 2: The framework of DARLC consists of two components: the representation learning and the deep clustering. The representation learning component integrates MIM and CL to generate image embeddings, which are then normalized through a non-linear projection head. Normalized representations are modeled by an SMM to derive their soft cluster assignments, which are used to construct various loss functions. With these loss functions, the two components are jointly optimized in a self-paced and end-to-end manner.
  • Figure 3: SGEPs from clusters generated by DARLC in dataset (ST-hLDPFC-{5-6}) with high and medium intra-cluster similarity.