Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Environmental Claim Detection with Hyperbolic Graph Neural Networks

Darpan Aswal, Manjira Sinha

TL;DR

This paper tackles environmental claim detection by moving away from heavyweight transformers to a graph-based framework that treats sentences as dependency graphs. It uses word2vec-based node features augmented with learnable POS embeddings, and encodes syntactic relations as edge features, with a weighted loss to handle data imbalance; classification is performed by GNNs and Hyperbolic GNNs in Euclidean and non-Euclidean spaces. The key finding is that hyperbolic graphs, especially in the Poincaré space with POS augmentation, achieve competitive or superior performance while using roughly 12M parameters (about 30x fewer than large transformers like RoBERTalarge), underscoring the value of explicit hierarchical modeling for NLP tasks. The results demonstrate the practical impact of hyperbolic geometry in NLP, offering a scalable, interpretable, and efficient alternative for specialized detection tasks and motivating broader exploration of non-Euclidean representations in language processing.

Abstract

Transformer based models, especially large language models (LLMs) dominate the field of NLP with their mass adoption in tasks such as text generation, summarization and fake news detection. These models offer ease of deployment and reliability for most applications, however, they require significant amounts of computational power for training as well as inference. This poses challenges in their adoption in resource-constrained applications, especially in the open-source community where compute availability is usually scarce. This work proposes a graph-based approach for Environmental Claim Detection, exploring Graph Neural Networks (GNNs) and Hyperbolic Graph Neural Networks (HGNNs) as lightweight yet effective alternatives to transformer-based models. Re-framing the task as a graph classification problem, we transform claim sentences into dependency parsing graphs, utilizing a combination of word2vec \& learnable part-of-speech (POS) tag embeddings for the node features and encoding syntactic dependencies in the edge relations. Our results show that our graph-based models, particularly HGNNs in the poincaré space (P-HGNNs), achieve performance superior to the state-of-the-art on environmental claim detection while using up to \textbf{30x fewer parameters}. We also demonstrate that HGNNs benefit vastly from explicitly modeling data in hierarchical (tree-like) structures, enabling them to significantly improve over their euclidean counterparts.

Efficient Environmental Claim Detection with Hyperbolic Graph Neural Networks

TL;DR

This paper tackles environmental claim detection by moving away from heavyweight transformers to a graph-based framework that treats sentences as dependency graphs. It uses word2vec-based node features augmented with learnable POS embeddings, and encodes syntactic relations as edge features, with a weighted loss to handle data imbalance; classification is performed by GNNs and Hyperbolic GNNs in Euclidean and non-Euclidean spaces. The key finding is that hyperbolic graphs, especially in the Poincaré space with POS augmentation, achieve competitive or superior performance while using roughly 12M parameters (about 30x fewer than large transformers like RoBERTalarge), underscoring the value of explicit hierarchical modeling for NLP tasks. The results demonstrate the practical impact of hyperbolic geometry in NLP, offering a scalable, interpretable, and efficient alternative for specialized detection tasks and motivating broader exploration of non-Euclidean representations in language processing.

Abstract

Transformer based models, especially large language models (LLMs) dominate the field of NLP with their mass adoption in tasks such as text generation, summarization and fake news detection. These models offer ease of deployment and reliability for most applications, however, they require significant amounts of computational power for training as well as inference. This poses challenges in their adoption in resource-constrained applications, especially in the open-source community where compute availability is usually scarce. This work proposes a graph-based approach for Environmental Claim Detection, exploring Graph Neural Networks (GNNs) and Hyperbolic Graph Neural Networks (HGNNs) as lightweight yet effective alternatives to transformer-based models. Re-framing the task as a graph classification problem, we transform claim sentences into dependency parsing graphs, utilizing a combination of word2vec \& learnable part-of-speech (POS) tag embeddings for the node features and encoding syntactic dependencies in the edge relations. Our results show that our graph-based models, particularly HGNNs in the poincaré space (P-HGNNs), achieve performance superior to the state-of-the-art on environmental claim detection while using up to \textbf{30x fewer parameters}. We also demonstrate that HGNNs benefit vastly from explicitly modeling data in hierarchical (tree-like) structures, enabling them to significantly improve over their euclidean counterparts.

Paper Structure

This paper contains 25 sections, 9 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Dependency Graph Construction
  5. Graph Representation
  6. Node Features
  7. Edge Features
  8. Integrating POS-tags with Node Features
  9. Weighted Loss for Imbalanced Data
  10. Graph-based Claim Classification
  11. Experimental Setup
  12. Dataset
  13. Models
  14. Evaluation Metrics
  15. Results & Observations
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Overview of the Graph-based Claim Detection Pipeline. Step 1: Claim sentence to dependency graph conversion. Steps 2: Dependencies are one-hot encoded as edge features. Node features are initialized with pretrained embeddings. Step3: Node features are concatenated with POS-tag embeddings learned by embedding layer. Step 4: Graph classification using a GNN/HGNN architecture trained with a weighted loss function.
  • Figure 2: The transformation of the example claim into a dependency graph. The graph shows tokens and their POS tags as nodes, with syntactic dependencies as labeled, directed edges.