EdgeMLOps: Operationalizing ML models with Cumulocity IoT and thin-edge.io for Visual quality Inspection
Kanishk Chaturvedi, Johannes Gasthuber, Mohamed Abdelaal
TL;DR
EdgeMLOps tackles the difficulty of running ML inference on constrained edge devices in industrial settings by combining Cumulocity IoT for centralized device management with thin-edge.io for lightweight edge deployment, and ONNX/ONNX Runtime for cross-platform inference. The authors present an architectural framework, demonstrate a Visual Quality Inspection use case on TTPLA data, and benchmark quantization methods on a Raspberry Pi 4, showing substantial inference speedups and model size reductions with minimal accuracy loss. The work provides a practical pathway for scalable, real-time edge AI in asset management, including OTA updates, data-driven model refreshes, and feedback loops for continuous improvement. Overall, EdgeMLOps demonstrates that efficient, maintainable AI deployment at the edge is feasible for complex industrial tasks, enabling faster maintenance decisions and improved operational efficiency.
Abstract
This paper introduces EdgeMLOps, a framework leveraging Cumulocity IoT and thin-edge.io for deploying and managing machine learning models on resource-constrained edge devices. We address the challenges of model optimization, deployment, and lifecycle management in edge environments. The framework's efficacy is demonstrated through a visual quality inspection (VQI) use case where images of assets are processed on edge devices, enabling real-time condition updates within an asset management system. Furthermore, we evaluate the performance benefits of different quantization methods, specifically static and dynamic signed-int8, on a Raspberry Pi 4, demonstrating significant inference time reductions compared to FP32 precision. Our results highlight the potential of EdgeMLOps to enable efficient and scalable AI deployments at the edge for industrial applications.