Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Language Models for Robust Greenwashing Detection

Neil Heinrich Braun, Keane Ong, Rui Mao, Erik Cambria, Gianmarco Mengaldo

TL;DR

This work tackles robustness gaps in ESG greenwashing detection by structuring latent representations through a parameter-efficient framework that jointly employs contrastive learning and ordinal ranking, augmented with gated feature modulation and MetaGradNorm for stable multi-objective optimization. By applying this approach to LoRA adapters across multiple open-source LLMs on the A3CG dataset, it demonstrates improved cross-category generalization and robustness over strong baselines, while highlighting that model scale alone does not guarantee better transfer. The key contributions include an integrated PEFT pipeline with contrastive+ordinal objectives, a per-sample gating mechanism, and a stabilized optimization strategy, validated through extensive experiments across models and categories. The findings have practical implications for robust ESG claim analysis and point to future work on hyperparameter stability, transferability, and broader multilingual generalization.

Abstract

Sustainability reports are critical for ESG assessment, yet greenwashing and vague claims often undermine their reliability. Existing NLP models lack robustness to these practices, typically relying on surface-level patterns that generalize poorly. We propose a parameter-efficient framework that structures LLM latent spaces by combining contrastive learning with an ordinal ranking objective to capture graded distinctions between concrete actions and ambiguous claims. Our approach incorporates gated feature modulation to filter disclosure noise and utilizes MetaGradNorm to stabilize multi-objective optimization. Experiments in cross-category settings demonstrate superior robustness over standard baselines while revealing a trade-off between representational rigidity and generalization.

Enhancing Language Models for Robust Greenwashing Detection

TL;DR

This work tackles robustness gaps in ESG greenwashing detection by structuring latent representations through a parameter-efficient framework that jointly employs contrastive learning and ordinal ranking, augmented with gated feature modulation and MetaGradNorm for stable multi-objective optimization. By applying this approach to LoRA adapters across multiple open-source LLMs on the A3CG dataset, it demonstrates improved cross-category generalization and robustness over strong baselines, while highlighting that model scale alone does not guarantee better transfer. The key contributions include an integrated PEFT pipeline with contrastive+ordinal objectives, a per-sample gating mechanism, and a stabilized optimization strategy, validated through extensive experiments across models and categories. The findings have practical implications for robust ESG claim analysis and point to future work on hyperparameter stability, transferability, and broader multilingual generalization.

Abstract

Sustainability reports are critical for ESG assessment, yet greenwashing and vague claims often undermine their reliability. Existing NLP models lack robustness to these practices, typically relying on surface-level patterns that generalize poorly. We propose a parameter-efficient framework that structures LLM latent spaces by combining contrastive learning with an ordinal ranking objective to capture graded distinctions between concrete actions and ambiguous claims. Our approach incorporates gated feature modulation to filter disclosure noise and utilizes MetaGradNorm to stabilize multi-objective optimization. Experiments in cross-category settings demonstrate superior robustness over standard baselines while revealing a trade-off between representational rigidity and generalization.
Paper Structure (53 sections, 16 equations, 5 figures, 12 tables, 2 algorithms)

This paper contains 53 sections, 16 equations, 5 figures, 12 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. NLP for ESG Analysis and Greenwashing
  4. Parameter-Efficient Fine-Tuning for LLMs.
  5. Contrastive Learning for NLP Representation Learning
  6. Ordinal and Structured Objectives in NLP
  7. Gating Mechanisms for Feature Modulation
  8. Multi-Objective Optimization and Loss Balancing
  9. Methodology
  10. Task Definition and Problem Setup
  11. Embedding Extraction and Similarity Functions
  12. Parameter-Efficient Adaptation with LoRA
  13. Contrastive Representation Learning
  14. Ordinal Ranking Objective
  15. Per-Sample Gating Mechanism
  16. ...and 38 more sections

Figures (5)

  • Figure 1: Seen vs Unseen vs $|\Delta|$ for Fold 1. Markers denote models; colors denote configurations.
  • Figure 2: Best configurations comparison across folds for seen (solid) and unseen (hatched) categories. Each fold shows the F1 scores for the best-performing model on seen categories paired with its unseen performance, highlighting the generalization gap across different architectures.
  • Figure 3: PCA and UMAP projections of latent representations for Mistral-7B on Fold 2. Left: Baseline shows diffuse clusters with low separation. Right: Full framework yields tighter, better-separated clusters by aspect category.
  • Figure 4: Seen vs. Unseen vs. $|\Delta|$ across all folds. Left: LoRA-based LLMs; Right: T5 full fine-tuning. Colors denote configurations; markers denote model backbones.
  • Figure 5: PCA and UMAP visualizations for Gemma-7B (Fold 1, top) and T5 (Fold 3, bottom). Baselines show diffuse clusters; full frameworks yield tighter separation. Sil.=Silhouette, CH=Calinski-Harabasz, Sep.=Separation Ratio.