Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contrastive Learning-Enhanced Large Language Models for Monolith-to-Microservice Decomposition

Khaled Sellami, Mohamed Aymen Saied

TL;DR

This work targets the costly monolith-to-microservice migration by introducing MonoEmbed, a two-phase approach that uses language-model embeddings of monolithic components and contrastive learning to optimize representations for decomposition. The Analysis phase trains various pre-trained models (including encoder-, decoder-, and LM-based embeddings) with a triplet loss and LoRA fine-tuning to produce microservice-aligned vectors; the Inference phase clusters these vectors to generate microservice partitions. Across eight monolithic applications and multiple benchmarks, MonoEmbed, particularly with ME-llm2vec and Affinity Propagation, achieves cohesive, balanced decompositions and outperforms existing methods in aggregate SCORE, demonstrating the practical impact of transformer-based embeddings for software architecture reorganization. The results underscore the potential for integrating LM-based representations into decomposition workflows and point to avenues for expanding granularity and multimodal inputs in future work.

Abstract

As Monolithic applications evolve, they become increasingly difficult to maintain and improve, leading to scaling and organizational issues. The Microservices architecture, known for its modularity, flexibility and scalability, offers a solution for large-scale applications allowing them to adapt and meet the demand on an ever increasing user base. Despite its advantages, migrating from a monolithic to a microservices architecture is often costly and complex, with the decomposition step being a significant challenge. This research addresses this issue by introducing MonoEmbed, a Language Model based approach for automating the decomposition process. MonoEmbed leverages state-of-the-art Large Language Models (LLMs) and representation learning techniques to generate representation vectors for monolithic components, which are then clustered to form microservices. By evaluating various pre-trained models and applying fine-tuning techniques such as Contrastive Learning and Low Rank Adaptation (LoRA), MonoEmbed aims to optimize these representations for microservice partitioning. The evaluation of the fine-tuned models showcases that they were able to significantly improve the quality of the representation vectors when compared with pre-trained models and traditional representations. The proposed approach was benchmarked against existing decomposition methods, demonstrating superior performance in generating cohesive and balanced microservices for monolithic applications with varying scales.

Contrastive Learning-Enhanced Large Language Models for Monolith-to-Microservice Decomposition

TL;DR

This work targets the costly monolith-to-microservice migration by introducing MonoEmbed, a two-phase approach that uses language-model embeddings of monolithic components and contrastive learning to optimize representations for decomposition. The Analysis phase trains various pre-trained models (including encoder-, decoder-, and LM-based embeddings) with a triplet loss and LoRA fine-tuning to produce microservice-aligned vectors; the Inference phase clusters these vectors to generate microservice partitions. Across eight monolithic applications and multiple benchmarks, MonoEmbed, particularly with ME-llm2vec and Affinity Propagation, achieves cohesive, balanced decompositions and outperforms existing methods in aggregate SCORE, demonstrating the practical impact of transformer-based embeddings for software architecture reorganization. The results underscore the potential for integrating LM-based representations into decomposition workflows and point to avenues for expanding granularity and multimodal inputs in future work.

Abstract

As Monolithic applications evolve, they become increasingly difficult to maintain and improve, leading to scaling and organizational issues. The Microservices architecture, known for its modularity, flexibility and scalability, offers a solution for large-scale applications allowing them to adapt and meet the demand on an ever increasing user base. Despite its advantages, migrating from a monolithic to a microservices architecture is often costly and complex, with the decomposition step being a significant challenge. This research addresses this issue by introducing MonoEmbed, a Language Model based approach for automating the decomposition process. MonoEmbed leverages state-of-the-art Large Language Models (LLMs) and representation learning techniques to generate representation vectors for monolithic components, which are then clustered to form microservices. By evaluating various pre-trained models and applying fine-tuning techniques such as Contrastive Learning and Low Rank Adaptation (LoRA), MonoEmbed aims to optimize these representations for microservice partitioning. The evaluation of the fine-tuned models showcases that they were able to significantly improve the quality of the representation vectors when compared with pre-trained models and traditional representations. The proposed approach was benchmarked against existing decomposition methods, demonstrating superior performance in generating cohesive and balanced microservices for monolithic applications with varying scales.

Paper Structure

This paper contains 53 sections, 1 equation, 5 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivating Example
  3. Background and Related Work
  4. Background
  5. Embedding Models (EMs)
  6. Language Models (LMs)
  7. Transformers
  8. Contrastive Learning (CL)
  9. Related Work
  10. Methodology
  11. Overview
  12. Model Selection
  13. Static and RNN based EMs
  14. Encoder-only Transformers (ET)
  15. Encoder-Decoder Transformers (EDT)
  16. ...and 38 more sections

Figures (5)

  • Figure 1: A UMAP 2-dimensional projection of calls, interactions and bag-of-words representations.
  • Figure 2: An overview of the suggested decomposition approach.
  • Figure 3: An example of Contrastive Learning training.
  • Figure 4: Evaluation $F_\beta$ scores for each clustering algorithm.
  • Figure 5: A UMAP 2-dimensional projection of VoyageAI, MonoEmbed-unixcoder and MonoEmbed-llm2vec embeddings.