Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing SDG-Text Classification with Combinatorial Fusion Analysis and Generative AI

Jingyan Xu, Marcelo L. LaFleur, Christina Schweikert, D. Frank Hsu

TL;DR

This work tackles SDG-text classification by fusing multiple, SDG-aware base models through Combinatorial Fusion Analysis (CFA) that leverages Rank-Score Characteristics and Cognitive Diversity. It augments limited labeled data with generative AI to train diverse classifiers and evaluates 104 fused configurations across 306 documents, outperforming the best single model with an average precision@1 of 0.9673. The results also reveal how CFA and human experts can complement each other, though disagreements underscore SDG interdependencies and motivate multi-label approaches. Limitations include tied rankings, limited available base models, and synthetic data distribution, guiding future work toward multi-label CFA, scalable MC-FCA, and richer data sources. Overall, CFA provides a domain-agnostic, robust ensemble framework that can extend SDG classification by integrating multiple intelligent sources and human judgment.

Abstract

(Natural Language Processing) NLP techniques such as text classification and topic discovery are very useful in many application areas including information retrieval, knowledge discovery, policy formulation, and decision-making. However, it remains a challenging problem in cases where the categories are unavailable, difficult to differentiate, or are interrelated. Social analysis with human context is an area that can benefit from text classification, as it relies substantially on text data. The focus of this paper is to enhance the classification of text according to the UN's Sustainable Development Goals (SDGs) by collecting and combining intelligence from multiple models. Combinatorial Fusion Analysis (CFA), a system fusion paradigm using a rank-score characteristic (RSC) function and cognitive diversity (CD), has been used to enhance classifier methods by combining a set of relatively good and mutually diverse classification models. We use a generative AI model to generate synthetic data for model training and then apply CFA to this classification task. The CFA technique achieves 96.73% performance, outperforming the best individual model. We compare the outcomes with those obtained from human domain experts. It is demonstrated that combining intelligence from multiple ML/AI models using CFA and getting input from human experts can, not only complement, but also enhance each other.

Enhancing SDG-Text Classification with Combinatorial Fusion Analysis and Generative AI

TL;DR

This work tackles SDG-text classification by fusing multiple, SDG-aware base models through Combinatorial Fusion Analysis (CFA) that leverages Rank-Score Characteristics and Cognitive Diversity. It augments limited labeled data with generative AI to train diverse classifiers and evaluates 104 fused configurations across 306 documents, outperforming the best single model with an average precision@1 of 0.9673. The results also reveal how CFA and human experts can complement each other, though disagreements underscore SDG interdependencies and motivate multi-label approaches. Limitations include tied rankings, limited available base models, and synthetic data distribution, guiding future work toward multi-label CFA, scalable MC-FCA, and richer data sources. Overall, CFA provides a domain-agnostic, robust ensemble framework that can extend SDG classification by integrating multiple intelligent sources and human judgment.

Abstract

(Natural Language Processing) NLP techniques such as text classification and topic discovery are very useful in many application areas including information retrieval, knowledge discovery, policy formulation, and decision-making. However, it remains a challenging problem in cases where the categories are unavailable, difficult to differentiate, or are interrelated. Social analysis with human context is an area that can benefit from text classification, as it relies substantially on text data. The focus of this paper is to enhance the classification of text according to the UN's Sustainable Development Goals (SDGs) by collecting and combining intelligence from multiple models. Combinatorial Fusion Analysis (CFA), a system fusion paradigm using a rank-score characteristic (RSC) function and cognitive diversity (CD), has been used to enhance classifier methods by combining a set of relatively good and mutually diverse classification models. We use a generative AI model to generate synthetic data for model training and then apply CFA to this classification task. The CFA technique achieves 96.73% performance, outperforming the best individual model. We compare the outcomes with those obtained from human domain experts. It is demonstrated that combining intelligence from multiple ML/AI models using CFA and getting input from human experts can, not only complement, but also enhance each other.
Paper Structure (16 sections, 6 equations, 8 figures, 4 tables)

This paper contains 16 sections, 6 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methods
  4. Combinatorial Fusion Analysis
  5. Base Models
  6. SDG Classy (Model A)
  7. LinkedSDG (Model B)
  8. SDG Mapper (Model C)
  9. Convolutional Neural Network (Model D)
  10. Random Forest (Model E)
  11. Experiment Workflow and Datasets Overview
  12. Results
  13. Combined Models Results
  14. Documents for which CFA and Best Base Models Disagrees
  15. Documents for which CFA and Best Base Models Disagree with human experts
  16. ...and 1 more sections

Figures (8)

  • Figure 1: SDGs of the United Nations UN_picture
  • Figure 2: RSC function graph w.r.t. models A, B, C, D, and E for document indicator 2-1-1
  • Figure 3: Step 1 of the Methodology Workflow
  • Figure 4: Step 2 of the Methodology Workflow
  • Figure 5: Step 3 of the Methodology Workflow
  • ...and 3 more figures