LoCoML: A Framework for Real-World ML Inference Pipelines
Kritin Maddireddy, Santhosh Kotekal Methukula, Chandrasekar Sridhar, Karthik Vaidhyanathan
TL;DR
LoCoML addresses the challenge of real-world ML inference integration across heterogeneous models in large collaborative settings. It introduces a low-code framework with a Model Hub and Pipeline Orchestrator to store models and compose pipelines, including adapters and graph-based validation. Evaluation shows linear overhead with increasing model count and minimal impact on overall inference time, enabling scalable, multi-component language services in Bhashini. This work demonstrates the practicality of model-driven engineering and low-code approaches for collaborative ML deployment.
Abstract
The widespread adoption of machine learning (ML) has brought forth diverse models with varying architectures, and data requirements, introducing new challenges in integrating these systems into real-world applications. Traditional solutions often struggle to manage the complexities of connecting heterogeneous models, especially when dealing with varied technical specifications. These limitations are amplified in large-scale, collaborative projects where stakeholders contribute models with different technical specifications. To address these challenges, we developed LoCoML, a low-code framework designed to simplify the integration of diverse ML models within the context of the \textit{Bhashini Project} - a large-scale initiative aimed at integrating AI-driven language technologies such as automatic speech recognition, machine translation, text-to-speech, and optical character recognition to support seamless communication across more than 20 languages. Initial evaluations show that LoCoML adds only a small amount of computational load, making it efficient and effective for large-scale ML integration. Our practical insights show that a low-code approach can be a practical solution for connecting multiple ML models in a collaborative environment.