Interactive Training: Feedback-Driven Neural Network Optimization

TL;DR

The paper addresses the rigidity of traditional fixed-training pipelines by introducing Interactive Training, a framework that enables real-time, feedback-driven interventions during neural network training via a control server, an interactive trainer, and a frontend dashboard. It demonstrates three case studies—human-in-the-loop, LLM-in-the-loop, and real-time training data updates—showing improved stability, reduced sensitivity to initial hyperparameters, and deployment-time adaptability. The approach leverages runtime callbacks, two-way communication, and branching training trajectories to turn training into an active process guided by human or AI agents, with open-source tooling built on Hugging Face Transformers. This work sets the stage for autonomous AI agents to monitor training logs and proactively intervene, promising more robust, efficient, and adaptable training workflows in practice.

Abstract

Traditional neural network training typically follows fixed, predefined optimization recipes, lacking the flexibility to dynamically respond to instabilities or emerging training issues. In this paper, we introduce Interactive Training, an open-source framework that enables real-time, feedback-driven intervention during neural network training by human experts or automated AI agents. At its core, Interactive Training uses a control server to mediate communication between users or agents and the ongoing training process, allowing users to dynamically adjust optimizer hyperparameters, training data, and model checkpoints. Through three case studies, we demonstrate that Interactive Training achieves superior training stability, reduced sensitivity to initial hyperparameters, and improved adaptability to evolving user needs, paving the way toward a future training paradigm where AI agents autonomously monitor training logs, proactively resolve instabilities, and optimize training dynamics.

Figures (8)

• Figure 1: Analogy of static vs. interactive training. Static training is like baking with a closed oven: hyperparameters follow a fixed, predetermined schedule until the end. Interactive training is like cooking on a stove: hyperparameters can be adjusted in real time.
• Figure 2: Interactive Training Frontend Dashboard. The left panel provides control tabs organized by Optimizer, Model, Checkpoint, and Dataset, allowing users to dynamically send intervention commands during training (e.g., adjusting the learning rate via the Optimizer panel shown). The right side displays real-time visualizations of training metrics, such as loss and gradient norm. Unlike traditional monitoring tools, this interface supports active two-way communication, enabling users to directly intervene and influence ongoing training processes in real-time.
• Figure 3: System Architecture. Users interact through a React-based Frontend Dashboard, which visualizes training metrics and sends control commands via REST API. The FastAPI-based Control Server mediates communication by forwarding user commands through command queues to the Interactive Trainer, implemented on top of Hugging Face's Trainer class. The trainer applies received commands via callback functions and sends real-time training updates back to the Control Server, which then broadcasts them to the Frontend Dashboard through WebSockets.
• Figure 4: Code changes required to enable Interactive Training.
• Figure 5: Comparison of human-in-the-loop Interactive Training versus traditional static training for finetuning GPT-2 on Wikitext-2. (a) Validation losses. Dynamic human interventions lead to improved optimization compared to the static baseline, which uses a fixed learning rate schedule. (b) Actual learning rates used over steps.