Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SemCSE-Multi: Multifaceted and Decodable Embeddings for Aspect-Specific and Interpretable Scientific Domain Mapping

Marc Brinner, Sina Zarrieß

TL;DR

The paper tackles interpretability and controllability gaps in scientific literature embeddings by introducing SemCSE-Multi, a framework that learns multiple, aspect-specific embeddingsunsupervisedly. It then distills these into a unified model capable of predicting all aspect embeddings from a full abstract and adds an embedding decoding pipeline that maps vectors back to natural language descriptions, enhancing interpretability. Extensive experiments in invasion biology demonstrate strong, aspect-disentangled similarity, while a medical-domain evaluation shows both promise and prompt-sensitivity challenges, underscoring the method's adaptability. Collectively, the work enables user-driven, interpretable domain mapping and interactive visualization for large-scale scientific corpora, with potential applicability beyond text to other modalities.

Abstract

We propose SemCSE-Multi, a novel unsupervised framework for generating multifaceted embeddings of scientific abstracts, evaluated in the domains of invasion biology and medicine. These embeddings capture distinct, individually specifiable aspects in isolation, thus enabling fine-grained and controllable similarity assessments as well as adaptive, user-driven visualizations of scientific domains. Our approach relies on an unsupervised procedure that produces aspect-specific summarizing sentences and trains embedding models to map semantically related summaries to nearby positions in the embedding space. We then distill these aspect-specific embedding capabilities into a unified embedding model that directly predicts multiple aspect embeddings from a scientific abstract in a single, efficient forward pass. In addition, we introduce an embedding decoding pipeline that decodes embeddings back into natural language descriptions of their associated aspects. Notably, we show that this decoding remains effective even for unoccupied regions in low-dimensional visualizations, thus offering vastly improved interpretability in user-centric settings.

SemCSE-Multi: Multifaceted and Decodable Embeddings for Aspect-Specific and Interpretable Scientific Domain Mapping

TL;DR

The paper tackles interpretability and controllability gaps in scientific literature embeddings by introducing SemCSE-Multi, a framework that learns multiple, aspect-specific embeddingsunsupervisedly. It then distills these into a unified model capable of predicting all aspect embeddings from a full abstract and adds an embedding decoding pipeline that maps vectors back to natural language descriptions, enhancing interpretability. Extensive experiments in invasion biology demonstrate strong, aspect-disentangled similarity, while a medical-domain evaluation shows both promise and prompt-sensitivity challenges, underscoring the method's adaptability. Collectively, the work enables user-driven, interpretable domain mapping and interactive visualization for large-scale scientific corpora, with potential applicability beyond text to other modalities.

Abstract

We propose SemCSE-Multi, a novel unsupervised framework for generating multifaceted embeddings of scientific abstracts, evaluated in the domains of invasion biology and medicine. These embeddings capture distinct, individually specifiable aspects in isolation, thus enabling fine-grained and controllable similarity assessments as well as adaptive, user-driven visualizations of scientific domains. Our approach relies on an unsupervised procedure that produces aspect-specific summarizing sentences and trains embedding models to map semantically related summaries to nearby positions in the embedding space. We then distill these aspect-specific embedding capabilities into a unified embedding model that directly predicts multiple aspect embeddings from a scientific abstract in a single, efficient forward pass. In addition, we introduce an embedding decoding pipeline that decodes embeddings back into natural language descriptions of their associated aspects. Notably, we show that this decoding remains effective even for unoccupied regions in low-dimensional visualizations, thus offering vastly improved interpretability in user-centric settings.

Paper Structure

This paper contains 41 sections, 11 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Training Aspect Embedding Models
  4. Motivation
  5. Dataset Creation
  6. Model Training
  7. Evaluation Setup
  8. Results
  9. Distilling a Unified Embedding Model
  10. Model Training
  11. Evaluation Setup
  12. Results
  13. Embedding Decoding
  14. Model Training
  15. Evaluation Setup
  16. ...and 26 more sections

Figures (4)

  • Figure 1: t-SNE visualizations of the embedding spaces of the unified embedding model for the hypothesis and species aspects, as well as for the SciNCL model ostendorff_neighborhood_2022. Each dot represents a scientific study from the test set, with colors indicating the corresponding human-annotated hypothesis labels.
  • Figure 2: The prompt provided to Mistral Small 3.1 (24B) for generating summarizing sentences for the aspect Hypothesis (general) within the field of Invasion Biology.
  • Figure 3: Aspect-specific summaries and generated reconstructions for corresponding aspect-embeddings of SemCSE-Multi. The exemplary abstract by invbioabstract is included in the test set.
  • Figure 4: The prompt provided to Qwen3-30B-A3B-Instruct-2507 for creating pairwise similarity assessments between two scientific abstracts for the hypothesis aspect.