Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Demystify Adult Learning: A Social Network and Large Language Model Assisted Approach

Fang Liu, Bosheng Ding, Chong Guan, Zhang Wei, Dusit Niyato, Justina Tan

TL;DR

This work addresses the need for accurate sentiment analysis in adult learning by leveraging social-network comments and domain-specific knowledge. It proposes A-Learn, a framework that labels comments with an LLM and then customizes base LLMs through transfer learning using an adult-learning dataset, achieving up to $91.3\%$ accuracy and a $\sim{16.8}\%$ average improvement over baselines. The method combines LLM-assisted labeling, selective layer fine-tuning, and evaluation on a expert-annotated test set, showing that domain adaptation yields substantial gains over generic sentiment analysis. Additionally, A-Learn enables insight extraction via word-cloud visualizations to identify core concerns and enablers in adult learning, informing teaching improvements and educational innovations.

Abstract

Adult learning is increasingly recognized as a crucial way for personal development and societal progress. It however is challenging, and adult learners face unique challenges such as balancing education with other life responsibilities. Collecting feedback from adult learners is effective in understanding their concerns and improving learning experiences, and social networks provide a rich source of real-time sentiment data from adult learners. Machine learning technologies especially large language models (LLMs) perform well in automating sentiment analysis. However, none of such models is specialized for adult learning with accurate sentiment understanding. In this paper, we present A-Learn, which enhances adult learning sentiment analysis by customizing existing general-purpose LLMs with domain-specific datasets for adult learning. We collect adult learners' comments from social networks and label the sentiment of each comment with an existing LLM to form labelled datasets tailored for adult learning. The datasets are used to customize A-Learn from several base LLMs. We conducted experimental studies and the results reveal A-Learn's competitive sentiment analysis performance, achieving up to 91.3% accuracy with 20% improvement over the base LLM. A-Learn is also employed for word cloud analysis to identify key concerns of adult learners. The research outcome of this study highlights the importance of applying machine learning with educational expertise for teaching improvement and educational innovations that benefit adult learning and adult learners.

Demystify Adult Learning: A Social Network and Large Language Model Assisted Approach

TL;DR

This work addresses the need for accurate sentiment analysis in adult learning by leveraging social-network comments and domain-specific knowledge. It proposes A-Learn, a framework that labels comments with an LLM and then customizes base LLMs through transfer learning using an adult-learning dataset, achieving up to accuracy and a average improvement over baselines. The method combines LLM-assisted labeling, selective layer fine-tuning, and evaluation on a expert-annotated test set, showing that domain adaptation yields substantial gains over generic sentiment analysis. Additionally, A-Learn enables insight extraction via word-cloud visualizations to identify core concerns and enablers in adult learning, informing teaching improvements and educational innovations.

Abstract

Adult learning is increasingly recognized as a crucial way for personal development and societal progress. It however is challenging, and adult learners face unique challenges such as balancing education with other life responsibilities. Collecting feedback from adult learners is effective in understanding their concerns and improving learning experiences, and social networks provide a rich source of real-time sentiment data from adult learners. Machine learning technologies especially large language models (LLMs) perform well in automating sentiment analysis. However, none of such models is specialized for adult learning with accurate sentiment understanding. In this paper, we present A-Learn, which enhances adult learning sentiment analysis by customizing existing general-purpose LLMs with domain-specific datasets for adult learning. We collect adult learners' comments from social networks and label the sentiment of each comment with an existing LLM to form labelled datasets tailored for adult learning. The datasets are used to customize A-Learn from several base LLMs. We conducted experimental studies and the results reveal A-Learn's competitive sentiment analysis performance, achieving up to 91.3% accuracy with 20% improvement over the base LLM. A-Learn is also employed for word cloud analysis to identify key concerns of adult learners. The research outcome of this study highlights the importance of applying machine learning with educational expertise for teaching improvement and educational innovations that benefit adult learning and adult learners.
Paper Structure (24 sections, 1 equation, 4 figures, 3 tables)

This paper contains 24 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. System Architecture
  3. LLM for Sentiment Analysis
  4. LLM Customization for Adult Learning
  5. Methodology
  6. LLM-assisted Sentiment Label Acquisition
  7. A-Learn with LLM Customization
  8. Experimental Study
  9. Dataset and Data Pre-processing
  10. Data Collection
  11. Data Pre-processing
  12. Datasets for Experiments
  13. LLM Labels for Training
  14. Expert Labels for Testing
  15. A-Learn Performance Analysis
  16. ...and 9 more sections

Figures (4)

  • Figure 1: The system architecture of A-Learn for adult learning sentiment analysis. A-Learn is ML-based and processes the social network comments as input to generate sentiment labels as output. A-Learn's sentiment analysis plays a crucial role in identifying key concerns and facilitating educational innovations in adult learning.
  • Figure 2: Sample comments with different emotions/sentiments about time management and teaching quality for adult learning.
  • Figure 3: An illustration of A-Learn with social network comments as the input. Domain knowledge is extracted from the comments and represented as sentiment initiates, used to fine-tune different layers in LLM's transformer encoder. The customized model can identify the sentiment of a comment.
  • Figure 4: The word clouds with highlighted adult learning concerns for both negative and positive sentiments. Larger words appear more frequently in the comments and imply significant concerns of adult learners.