Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning

TL;DR

The paper addresses the sensitivity of contrastive sentence representation learning to the quality of LLM-generated data. It introduces MultiCSR, a three-stage framework that explicitly refines generation (Stage 1), self-curation of sentence pairs (Stage 2), and in-batch training with a similarity mask (Stage 3), integrating these refinements with a base learner like SimCSE. Key contributions include decomposing LLM-based corpus construction for sentence embeddings, demonstrating that smaller open LLMs can outperform larger closed ones under refinement, and providing extensive STS and BEIR evaluations along with data-resource analyses. The results show significant improvements over strong baselines, indicating a scalable path to high-quality sentence representations across domains using LLMs.

Abstract

Recently, large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. Existing methods have explored utilizing LLMs as data annotators to generate synthesized data for training contrastive learning based sentence embedding models such as SimCSE. However, since contrastive learning models are sensitive to the quality of sentence pairs, the effectiveness of these methods is largely influenced by the content generated from LLMs, highlighting the need for more refined generation in the context of sentence representation learning. Building upon this premise, we propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus for training base sentence embedding models into three stages (i.e., sentence generation, sentence pair construction, in-batch training) and refines the generated content at these three distinct stages, ensuring only high-quality sentence pairs are utilized to train a base contrastive learning model. Our extensive experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results. Comprehensive analyses further underscore the potential of our framework in various application scenarios and achieving better sentence representation learning with LLMs.

Figures (7)

• Figure 1: Two example sentences with the generations from Flan-T5 and ChatGPT given different instructions.
• Figure 2: Overview of our three-stage framework MultiCSR. Stage 1: Contrastive Generation. We refine each token's logits with the opposite instruction to align more closely with the intended instruction. Stage 2: Contrastive Sentence Pair Construction. By prompting LLMs to evaluate the semantic similarities of generated sentence pairs, we ensure that only sentence pairs satisfying the pre-defined rules are left to form a curated set. Stage 3: Contrastive In-Batch Training. We leverage the similarity mask provided by a pre-trained sentence representation model to contrastively filter false negatives during the in-batch training.
• Figure 3: One example of our self-curation strategy. During the final training stage, we only use the sentence pairs in orange region.
• Figure 4: Performance comparison between with and without our constrastive in-batch training stage, with the number of duplications and Spearman's scores on the development set of STS-B reported.
• Figure 5: Generation improvement by incorporating noisy instruction with $l_t-\omega*\hat{l}_t$ into our contrastive generation stage.