Helios: A Foundational Language Model for Smart Energy Knowledge Reasoning and Application

TL;DR

Helios delivers the first open-source foundation LLM tailored for smart energy, addressing the domain-knowledge and physical-constraint gaps of general LLMs. It introduces EnerSys, a multi-agent data pipeline producing EnerBase, EnerInstruct, and EnerReinforce to support domain-specific pretraining, instruction tuning, and RLHF, plus EnerBench for rigorous evaluation. Across pretraining, instruction tuning with LoRA, and RLHF, Helios demonstrates superior domain knowledge and task performance relative to similar-scale models and approaches GPT-4 on several metrics, illustrating strong potential for practical smart-energy reasoning and coding tasks. The work emphasizes human-in-the-loop safety, expert validation, and transparent data pipelines, underscoring the model's role as an intelligent reference assistant rather than an autonomous decision-maker in high-stakes energy systems.

Abstract

In the global drive toward carbon neutrality, deeply coordinated smart energy systems underpin industrial transformation. However, the interdisciplinary, fragmented, and fast-evolving expertise in this domain prevents general-purpose LLMs, which lack domain knowledge and physical-constraint awareness, from delivering precise engineering-aligned inference and generation. To address these challenges, we introduce Helios, a large language model tailored to the smart energy domain, together with a comprehensive suite of resources to advance LLM research in this field. Specifically, we develop Enersys, a multi-agent collaborative framework for end-to-end dataset construction, through which we produce: (1) a smart energy knowledge base, EnerBase, to enrich the model's foundational expertise; (2) an instruction fine-tuning dataset, EnerInstruct, to strengthen performance on domain-specific downstream tasks; and (3) an RLHF dataset, EnerReinforce, to align the model with human preferences and industry standards. Leveraging these resources, Helios undergoes large-scale pretraining, SFT, and RLHF. We also release EnerBench, a benchmark for evaluating LLMs in smart energy scenarios, and demonstrate that our approach significantly enhances domain knowledge mastery, task execution accuracy, and alignment with human preferences.

Figures (5)

• Figure 1: Limitations of Scheduling Decision-Making in the Smart Energy Domain.
• Figure 2: The multi-agent collaboration framework EnerSys provides the data required for Helios' three-stage training, including pre-training data (EnerBase), instruction tuning data (EnerInstruct), and RLHF data (EnerReinforce).
• Figure 3: Text processed by the Parsing-Agent. A.Images: only the captions are retained, image bodies are removed; B.Tables: converted to Markdown format; C.Complex mathematical formulae: converted to Markdown format; D.Citations: for each citation, the corresponding page numbers of the referenced literature are specified.
• Figure 4: Dataset Quality Optimization workflow example. (A) Text before processing; (B) the Check‑Agent scores the text quality and provides optimization suggestions; (C) the Optimization‑Agent generates the optimized text based on those suggestions. We mark the differences in Red.
• Figure 5: Case analysis of modeling tasks in the smart energy domain.