Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multivocal Review of MLOps Practices, Challenges and Open Issues

Beyza Eken, Samodha Pallewatta, Nguyen Khoi Tran, Ayse Tosun, Muhammad Ali Babar

TL;DR

This paper addresses the fragmented state of MLOps knowledge by conducting a Multivocal Literature Review that fuses 150 peer-reviewed studies with 48 gray sources to produce a unified conceptualization of MLOps. It maps core definitions, lifecycle activities, and practitioner roles, and then catalogs state-of-the-art practices across team structure, transparency, monitoring, infrastructure provisioning, deployment, pipelines, and complexity management. It also identifies socio-technical, pipeline, and platform-related adoption challenges and proposes solutions such as domain-specific reference architectures, governance tooling, and human-in-the-loop approaches. The study highlights emerging needs for responsible AI, provenance, artifact governance, and scalable, secure platforms, offering a comprehensive roadmap for researchers and practitioners to advance MLOps adoption and maturity.

Abstract

MLOps has emerged as a key solution to address many socio-technical challenges of bringing ML models to production, such as integrating ML models with non-ML software, continuous monitoring, maintenance, and retraining of deployed models. Despite the utility of MLOps, an integrated body of knowledge regarding MLOps remains elusive because of its extensive scope due to the diversity of ML productionalization challenges it addresses. Whilst the existing literature reviews provide valuable snapshots of specific practices, tools, and research prototypes related to MLOps at various times, they focus on particular facets of MLOps, thus fail to offer a comprehensive and invariant framework that can weave these perspectives into a unified understanding of MLOps. This paper presents a Multivocal Literature Review that systematically analyzes a corpus of 150 peer-reviewed and 48 grey literature to synthesize a unified conceptualization of MLOps and develop a snapshot of its best practices, adoption challenges, and solutions.

A Multivocal Review of MLOps Practices, Challenges and Open Issues

TL;DR

This paper addresses the fragmented state of MLOps knowledge by conducting a Multivocal Literature Review that fuses 150 peer-reviewed studies with 48 gray sources to produce a unified conceptualization of MLOps. It maps core definitions, lifecycle activities, and practitioner roles, and then catalogs state-of-the-art practices across team structure, transparency, monitoring, infrastructure provisioning, deployment, pipelines, and complexity management. It also identifies socio-technical, pipeline, and platform-related adoption challenges and proposes solutions such as domain-specific reference architectures, governance tooling, and human-in-the-loop approaches. The study highlights emerging needs for responsible AI, provenance, artifact governance, and scalable, secure platforms, offering a comprehensive roadmap for researchers and practitioners to advance MLOps adoption and maturity.

Abstract

MLOps has emerged as a key solution to address many socio-technical challenges of bringing ML models to production, such as integrating ML models with non-ML software, continuous monitoring, maintenance, and retraining of deployed models. Despite the utility of MLOps, an integrated body of knowledge regarding MLOps remains elusive because of its extensive scope due to the diversity of ML productionalization challenges it addresses. Whilst the existing literature reviews provide valuable snapshots of specific practices, tools, and research prototypes related to MLOps at various times, they focus on particular facets of MLOps, thus fail to offer a comprehensive and invariant framework that can weave these perspectives into a unified understanding of MLOps. This paper presents a Multivocal Literature Review that systematically analyzes a corpus of 150 peer-reviewed and 48 grey literature to synthesize a unified conceptualization of MLOps and develop a snapshot of its best practices, adoption challenges, and solutions.
Paper Structure (63 sections, 4 figures, 3 tables)

This paper contains 63 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Planning the Review
  4. Study Search and Selection Protocol
  5. Automatic Search
  6. Primary Study Selection from Academic Literature
  7. Primary Study Selection from Gray Literature
  8. Data Extraction and Thematic Analysis
  9. Threats to Validity
  10. RQ1: Conceptualization of MLOps
  11. RQ1.1 - How is MLOps Defined in the Literature?
  12. MLOps-as-Continuous-ML
  13. DevOps-for-ML
  14. MLOps-as-Pipeline-Automation
  15. MLOps-as-Culture
  16. ...and 48 more sections

Figures (4)

  • Figure 1: Research Methodology
  • Figure 2: MLOps definitions and the ML productionalization problems they address
  • Figure 3: A pipeline for ML development and operations, and associated roles: Domain Experts (DME); Business Analysts (BA); Data Engineers (DE); Data Scientists (DS); ML Engineers (MLE); MLOps Engineers (MLOE); DevOps Engineers (DOE); Software Engineers (SE); Quality Assurance Engineers (QAE); Operational Engineers (OPE); Technical and Solution Architects (ARC); Security Experts (SE); Business Owners/Managers (BM); Product Owners/Managers/Experts (PM); Data Governance Officer (DGO), Data Quality Board (DQB), Data Steward (DST)
  • Figure 4: MLOps solutions and the corresponding ML productionalization challenges