Active Inference Meeting Energy-Efficient Control of Parallel and Identical Machines

TL;DR

This work addresses energy-efficient control (EEC) for parallel, identical machines in manufacturing by deploying a deep active inference (AIF) agent within a POMDP framework. The approach minimizes the Expected Free Energy ($EFE$) while calibrating its generative model through Variational Free Energy ($VFE$), enabling decision-making under uncertainty without full system dynamics. To tackle stochasticity and delayed policy response, the authors introduce multi-step transition planning and a hybrid horizon that blends long-horizon Q-learning priors with short-horizon $EFE$ planning, along with experience replay and extended planning horizons. Experimental results on a six-machine workstation show that these enhancements yield higher rewards and better discrimination between actions, highlighting the potential of AIF as a robust alternative to model-free RL in non-stationary, stochastic manufacturing settings. The findings suggest practical pathways for energy savings while maintaining throughput, and point to future work incorporating recurrent or diffusion-based generative models to further improve predictive capabilities and adaptability.

Abstract

We investigate the application of active inference in developing energy-efficient control agents for manufacturing systems. Active inference, rooted in neuroscience, provides a unified probabilistic framework integrating perception, learning, and action, with inherent uncertainty quantification elements. Our study explores deep active inference, an emerging field that combines deep learning with the active inference decision-making framework. Leveraging a deep active inference agent, we focus on controlling parallel and identical machine workstations to enhance energy efficiency. We address challenges posed by the problem's stochastic nature and delayed policy response by introducing tailored enhancements to existing agent architectures. Specifically, we introduce multi-step transition and hybrid horizon methods to mitigate the need for complex planning. Our experimental results demonstrate the effectiveness of these enhancements and highlight the potential of the active inference-based approach.

Figures (5)

• Figure 1: Layout of parallel and identical machines in the workstation LOFFREDO202391.
• Figure 2: The illustration depicts two views of the active inference framework: general steps on the left and active inference elements on the right.
• Figure 3: The agent's architecture and generative framework resemble that of a VAE. The line on top represents the agent simulating the future and making a prediction, while on the bottom, the agent receives a new observation after $\Delta _{t}$ of taking an action, $\tilde{a}_{t}$ .
• Figure 4: Comparison of test rewards during training of agents with 90-step transition against 1-step transition, when $\lambda_s=1$.
• Figure 5: The test performance of the agent with 90-step transition and repeated actions for planning, when $\lambda_s=1.5$ and $\gamma=0.05$, replicated with 10 different random seeds. A: Average reward. B: Average planner distribution for different actions. C: Average EFE's term 2 for different actions.