Identifying architectural design decisions for achieving green ML serving

TL;DR

This paper tackles the sustainability of ML inference by examining architectural design decisions (ADDs) in ML serving and their relation to quality attributes, including energy efficiency. Using seed papers from two reviews and forward snowballing, the authors collect 12 studies and apply thematic analysis aligned with ISO 25010 and a Green AI extension to map ADDs to quality factors. They identify four main serving infrastructures (SI1–SI4) and four transversal decisions (TD1–TD4), concluding that the Serving Infrastructure is the central ADD influencing system architecture and trade-offs. The study reveals a strong emphasis on performance efficiency with limited attention to energy consumption and other QoS aspects, highlighting substantial opportunities for energy-focused analysis and green-aware design guidance in future ML-serving research.

Abstract

The growing use of large machine learning models highlights concerns about their increasing computational demands. While the energy consumption of their training phase has received attention, fewer works have considered the inference phase. For ML inference, the binding of ML models to the ML system for user access, known as ML serving, is a critical yet understudied step for achieving efficiency in ML applications. We examine the literature in ML architectural design decisions and Green AI, with a special focus on ML serving. The aim is to analyze ML serving architectural design decisions for the purpose of understanding and identifying them with respect to quality characteristics from the point of view of researchers and practitioners in the context of ML serving literature. Our results (i) identify ML serving architectural design decisions along with their corresponding components and associated technological stack, and (ii) provide an overview of the quality characteristics studied in the literature, including energy efficiency. This preliminary study is the first step in our goal to achieve green ML serving. Our analysis may aid ML researchers and practitioners in making green-aware architecture design decisions when serving their models.

Figures (1)

• Figure 1: ML serving ADDs.