SmartPilot: A Multiagent CoPilot for Adaptive and Intelligent Manufacturing

TL;DR

SmartPilot introduces a neurosymbolic, multiagent CoPilot for Industry 4.0 that unifies anomaly prediction, production forecasting, and domain-specific Q&A. The framework deploys three edge-optimized agents—PredictX, ForeSight, and InfoGuide—coupled with ontology-informed knowledge infusion and multimodal data fusion to enable real-time, explainable decision support. Empirical results on rocket-assembly and Vegemite production demonstrate strong anomaly prediction accuracy (≈93%), substantial forecasting gains over LSTM baselines, and high-quality, speedy Q&A performance. This work advances practical, interpretable, and scalable AI-assisted manufacturing with end-to-end integration and real-time back-end connectivity.

Abstract

In the dynamic landscape of Industry 4.0, achieving efficiency, precision, and adaptability is essential to optimize manufacturing operations. Industries suffer due to supply chain disruptions caused by anomalies, which are being detected by current AI models but leaving domain experts uncertain without deeper insights into these anomalies. Additionally, operational inefficiencies persist due to inaccurate production forecasts and the limited effectiveness of traditional AI models for processing complex sensor data. Despite these advancements, existing systems lack the seamless integration of these capabilities needed to create a truly unified solution for enhancing production and decision-making. We propose SmartPilot, a neurosymbolic, multiagent CoPilot designed for advanced reasoning and contextual decision-making to address these challenges. SmartPilot processes multimodal sensor data and is compact to deploy on edge devices. It focuses on three key tasks: anomaly prediction, production forecasting, and domain-specific question answering. By bridging the gap between AI capabilities and real-world industrial needs, SmartPilot empowers industries with intelligent decision-making and drives transformative innovation in manufacturing. The demonstration video, datasets, and supplementary materials are available at https://github.com/ChathurangiShyalika/SmartPilot.

Figures (8)

• Figure 1: Agent-based architecture of SmartPilot and their interactions
• Figure 2: Architecture of PredictX agent: It integrates time series data and image inputs for anomaly prediction through a multi-stage process. The system begins with preprocessing and feature extraction, utilizing a pretrained EfficientNet (PC) model for image features and a time series autoencoder (TSA) for time series data. The extracted features are then fused, incorporating external process ontology knowledge to enhance the model's predictive capabilities. The fusion model ultimately predicts the next time series output and classifies the anomaly types. Three baseline approaches are implemented for comparison: P1, a Decision-Level Fusion approach; P2, a Decision-Level Fusion with Transfer Learning (where the hyperparameters of the autoencoder and EfficientNet-B0, training process, and loss function remain consistent with P1, but the encoder is frozen to prevent gradient updates); and P3, an Enhanced Decision-Level Fusion with Transfer Learning via Neurosymbolic AI. In P3, a custom loss function is introduced that combines Weighted Mean Squared Error (WMSE) loss with an additional penalty, infusing external knowledge on sensor ranges derived from the process ontology.
• Figure 3: Architecture of ForeSight agent: It utilizes an LSTM model for production forecasting, using historical data from the target variables. The architecture includes two LSTM layers to capture temporal dependencies, with additional features infused at the dense layer level.
• Figure 4: Architecture of InfoGuide agent: The blue area indicates the boundary of the InfoGuide agent. InfoGuide agent also interacts with the PredictX and ForeSight agents to respond to queries related to real-time anomaly prediction and production forecasting.
• Figure 5: Deployment Setup of SmartPilot