Machine Learning Systems are Bloated and Vulnerable

TL;DR

This work tackles the problem of bloated ML container deployments by introducing MMLB, a framework that measures container and package-level bloat, analyzes its sources, and evaluates associated vulnerabilities. By debloating 15 TensorFlow/PyTorch-based containers with a modified Cimplifier, the authors quantify that up to 80% of container size is bloat and demonstrate substantial provisioning-time improvements (up to 3.7×) and CVE reductions (up to 99%). The framework couples container-level metrics with package-dependency analysis via a package-attribute graph to reveal how APT/PIP/Conda dependencies propagate bloat and insecurity, highlighting that generic packages often drive CVEs more than ML packages. The findings advocate for leaner, modular ML packaging and intelligent, workload-aware container ecosystems to reduce resource waste and security risk in ML deployments.

Abstract

Today's software is bloated with both code and features that are not used by most users. This bloat is prevalent across the entire software stack, from operating systems and applications to containers. Containers are lightweight virtualization technologies used to package code and dependencies, providing portable, reproducible and isolated environments. For their ease of use, data scientists often utilize machine learning containers to simplify their workflow. However, this convenience comes at a cost: containers are often bloated with unnecessary code and dependencies, resulting in very large sizes. In this paper, we analyze and quantify bloat in machine learning containers. We develop MMLB, a framework for analyzing bloat in software systems, focusing on machine learning containers. MMLB measures the amount of bloat at both the container and package levels, quantifying the sources of bloat. In addition, MMLB integrates with vulnerability analysis tools and performs package dependency analysis to evaluate the impact of bloat on container vulnerabilities. Through experimentation with 15 machine learning containers from TensorFlow, PyTorch, and Nvidia, we show that bloat accounts for up to 80% of machine learning container sizes, increasing container provisioning times by up to 370% and exacerbating vulnerabilities by up to 99%.

Figures (17)

• Figure 1: ML code is only a small part of ML production systems.
• Figure 2: Stack-plots of the lines of code (LOC) added to TensorFlow and PyTorch over their lifetime. The $y-axis$ is the LoC. The $x-axis$ is time. The sudden drop in the LoC of TensorFlow in late 2019 is due to the release of TensorFlow 2.0, which removed a lot of code. The large jump in the LoC of PyTorch in 2018 is due to the merging of PyTorch and Caffe2.
• Figure 3: Sizes of ML containers versus generic containers. The cumulative distribution function (CDF) of container sizes is analyzed from the top 100 pulled generic containers and ML containers in DockerHub.
• Figure 4: MMLB Framework overview. MMLB selects various combinations of ML frameworks, ML tasks, ML models, and ML container images. MMLB then debloats the containers using a modified version of Cimplifier. The debloated containers are then analyzed using container-level analysis, package-level analysis, vulnerability analysis, and package dependency analysis to find the bloat degree, the sources of bloat, the vulnerabilities, and the package dependencies.
• Figure 5: Pareto charts of files in debloated generic containers and ML containers. The x-axis shows indices instead of filenames for simplicity.

Theorems & Definitions (6)

• Definition 3.1
• Definition 3.2
• Definition 3.3
• Definition 3.4
• Definition 3.5
• Definition 3.6