Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Graph-based RAG for Energy Efficiency Question Answering

Riccardo Campi, Nicolò Oreste Pinciroli Vago, Mathyas Giudici, Pablo Barrachina Rodriguez-Guisado, Marco Brambilla, Piero Fraternali

TL;DR

This paper introduces a graph-based Retrieval Augmented Generation (RAG) architecture tailored for Energy Efficiency (EE) Question Answering. It automatically extracts a knowledge graph from domain documents (regulations, incentives, and guidance), then uses local graph reasoning and multilingual generation to answer user queries with source citations. Validation via the RAGAs framework on 101 QAs across Italian and English shows an overall answer-validity of 75.2% ±2.7%, with higher performance on general EE questions and a 4.4% translation-related accuracy loss, underscoring the benefits and multilingual capabilities of graph-based RAG. The work demonstrates the potential to deploy such a system within EE decision-support tools like ENERGENIUS, while highlighting the need for broader multilingual testing and extended knowledge extraction processes.

Abstract

In this work, we investigate the use of Large Language Models (LLMs) within a graph-based Retrieval Augmented Generation (RAG) architecture for Energy Efficiency (EE) Question Answering. First, the system automatically extracts a Knowledge Graph (KG) from guidance and regulatory documents in the energy field. Then, the generated graph is navigated and reasoned upon to provide users with accurate answers in multiple languages. We implement a human-based validation using the RAGAs framework properties, a validation dataset comprising 101 question-answer pairs, and domain experts. Results confirm the potential of this architecture and identify its strengths and weaknesses. Validation results show how the system correctly answers in about three out of four of the cases (75.2 +- 2.7%), with higher results on questions related to more general EE answers (up to 81.0 +- 4.1%), and featuring promising multilingual abilities (4.4% accuracy loss due to translation).

A Graph-based RAG for Energy Efficiency Question Answering

TL;DR

This paper introduces a graph-based Retrieval Augmented Generation (RAG) architecture tailored for Energy Efficiency (EE) Question Answering. It automatically extracts a knowledge graph from domain documents (regulations, incentives, and guidance), then uses local graph reasoning and multilingual generation to answer user queries with source citations. Validation via the RAGAs framework on 101 QAs across Italian and English shows an overall answer-validity of 75.2% ±2.7%, with higher performance on general EE questions and a 4.4% translation-related accuracy loss, underscoring the benefits and multilingual capabilities of graph-based RAG. The work demonstrates the potential to deploy such a system within EE decision-support tools like ENERGENIUS, while highlighting the need for broader multilingual testing and extended knowledge extraction processes.

Abstract

In this work, we investigate the use of Large Language Models (LLMs) within a graph-based Retrieval Augmented Generation (RAG) architecture for Energy Efficiency (EE) Question Answering. First, the system automatically extracts a Knowledge Graph (KG) from guidance and regulatory documents in the energy field. Then, the generated graph is navigated and reasoned upon to provide users with accurate answers in multiple languages. We implement a human-based validation using the RAGAs framework properties, a validation dataset comprising 101 question-answer pairs, and domain experts. Results confirm the potential of this architecture and identify its strengths and weaknesses. Validation results show how the system correctly answers in about three out of four of the cases (75.2 +- 2.7%), with higher results on questions related to more general EE answers (up to 81.0 +- 4.1%), and featuring promising multilingual abilities (4.4% accuracy loss due to translation).

Paper Structure

This paper contains 22 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. LLM.
  4. RAG.
  5. Related work.
  6. Methodology
  7. Knowledge Extraction
  8. Knowledge Base
  9. Retrieval & Generation
  10. Validation experiment
  11. Validation dataset
  12. Experimental setup
  13. Results
  14. Ablation Experiment
  15. Discussion
  16. ...and 7 more sections

Figures (1)

  • Figure 1: General architecture of the proposed system. The system relies on 3 main parts: the Knowledge Extraction, the Knowledge Base, and the Retrieval & Generation.