FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data

TL;DR

FELA tackles the challenge of automated feature engineering for industrial event log data by introducing a multi-agent system that leverages LLMs to generate and realize high-quality, explainable features. The framework combines Idea, Code, Critic, and Evaluate agents within a hierarchical knowledge base and dual-memory memory to drive an agentic evolution algorithm that blends reinforcement learning and genetic principles, balancing exploration and exploitation via UCB. Empirical results on Dia, Taobao, and Tencent demonstrate improved predictive performance and steerable, interpretable feature generation across diverse domains, with ablations verifying the value of critics, memory, and the exploration strategy. The work suggests a practical path toward scalable, adaptable, and domain-aligned automated feature engineering for complex, real-world data streams.

Abstract

Event log data, recording fine-grained user actions and system events, represent one of the most valuable assets for modern digital services. However, the complexity and heterogeneity of industrial event logs--characterized by large scale, high dimensionality, diverse data types, and intricate temporal or relational structures--make feature engineering extremely challenging. Existing automatic feature engineering approaches, such as AutoML or genetic methods, often suffer from limited explainability, rigid predefined operations, and poor adaptability to complicated heterogeneous data. In this paper, we propose FELA (Feature Engineering LLM Agents), a multi-agent evolutionary system that autonomously extracts meaningful and high-performing features from complex industrial event log data. FELA integrates the reasoning and coding capabilities of large language models (LLMs) with an insight-guided self-evolution paradigm. Specifically, FELA employs specialized agents--Idea Agents, Code Agents, and Critic Agents--to collaboratively generate, validate, and implement novel feature ideas. An Evaluation Agent summarizes feedback and updates a hierarchical knowledge base and dual-memory system to enable continual improvement. Moreover, FELA introduces an agentic evolution algorithm, combining reinforcement learning and genetic algorithm principles to balance exploration and exploitation across the idea space. Extensive experiments on real industrial datasets demonstrate that FELA can generate explainable, domain-relevant features that significantly improve model performance while reducing manual effort. Our results highlight the potential of LLM-based multi-agent systems as a general framework for automated, interpretable, and adaptive feature engineering in complex real-world environments.

Figures (11)

• Figure 1: Human data scientists explore existing ideas to derive features, synthesize new concepts, and decide which idea to pursue next through logical reasoning and creative insight.
• Figure 2: System overview of FELA. The red arrows illustrate the core self-evolution loop of the system. The idea agents propose new feature concepts or experimental ideas, which are then translated into executable feature engineering code by the code agents. The generated code is executed and evaluated by the evaluate agent using real event log data to produce corresponding performance rewards. The resulting experiences are summarized and updated into both the short-term memory and the global long-term memory, guiding subsequent rounds of exploration. The idea critic and code critic modules enhance robustness and consistency by validating generated ideas and code, respectively. The best-performing features and implementations can be seamlessly exported to production environments.
• Figure 3: An illustration of the knowledge base. It contains rich information including idea insights, ucb scores, associated feature and corresponding pseudocodes, etc.
• Figure 4: Thinking paradigm of our agents. All agents follow this reasoning framework unless otherwise stated.
• Figure 5: Long Term Memory. Long term memory is updated in an adaptive manner along with the evolution iterations.