Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Persona Classification in Dialogue Systems: A Graph Neural Network Approach

Konstantin Zaitsev

TL;DR

This work addresses persona classification in dialogue systems by integrating text embeddings with Graph Neural Networks on a manually annotated dataset. It constructs a homogeneous graph of persona statements, where edges reflect semantic similarity via k-NN and NLI-based weighting, and uses a Graph Neural Network to propagate information for multi-label classification; final predictions combine encoder and GNN outputs as $Z = \lambda Z_{GNN} + (1 - \lambda) Z_{Encoder}$ with $\lambda = 0.7$. The key contributions are a practical framework and a manually labeled MSC-derived dataset, with experimental evidence that GNNs improve performance and data efficiency, particularly in low-resource settings. This approach supports more efficient and scalable personalization in dialogue systems and offers a pathway for future work on annotation robustness and dataset diversity.

Abstract

In recent years, Large Language Models (LLMs) gain considerable attention for their potential to enhance personalized experiences in virtual assistants and chatbots. A key area of interest is the integration of personas into LLMs to improve dialogue naturalness and user engagement. This study addresses the challenge of persona classification, a crucial component in dialogue understanding, by proposing a framework that combines text embeddings with Graph Neural Networks (GNNs) for effective persona classification. Given the absence of dedicated persona classification datasets, we create a manually annotated dataset to facilitate model training and evaluation. Our method involves extracting semantic features from persona statements using text embeddings and constructing a graph where nodes represent personas and edges capture their similarities. The GNN component uses this graph structure to propagate relevant information, thereby improving classification performance. Experimental results show that our approach, in particular the integration of GNNs, significantly improves classification performance, especially with limited data. Our contributions include the development of a persona classification framework and the creation of a dataset.

Enhancing Persona Classification in Dialogue Systems: A Graph Neural Network Approach

TL;DR

This work addresses persona classification in dialogue systems by integrating text embeddings with Graph Neural Networks on a manually annotated dataset. It constructs a homogeneous graph of persona statements, where edges reflect semantic similarity via k-NN and NLI-based weighting, and uses a Graph Neural Network to propagate information for multi-label classification; final predictions combine encoder and GNN outputs as with . The key contributions are a practical framework and a manually labeled MSC-derived dataset, with experimental evidence that GNNs improve performance and data efficiency, particularly in low-resource settings. This approach supports more efficient and scalable personalization in dialogue systems and offers a pathway for future work on annotation robustness and dataset diversity.

Abstract

In recent years, Large Language Models (LLMs) gain considerable attention for their potential to enhance personalized experiences in virtual assistants and chatbots. A key area of interest is the integration of personas into LLMs to improve dialogue naturalness and user engagement. This study addresses the challenge of persona classification, a crucial component in dialogue understanding, by proposing a framework that combines text embeddings with Graph Neural Networks (GNNs) for effective persona classification. Given the absence of dedicated persona classification datasets, we create a manually annotated dataset to facilitate model training and evaluation. Our method involves extracting semantic features from persona statements using text embeddings and constructing a graph where nodes represent personas and edges capture their similarities. The GNN component uses this graph structure to propagate relevant information, thereby improving classification performance. Experimental results show that our approach, in particular the integration of GNNs, significantly improves classification performance, especially with limited data. Our contributions include the development of a persona classification framework and the creation of a dataset.

Paper Structure

This paper contains 17 sections, 4 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data
  4. Dataset
  5. Persona Labeling
  6. Approach
  7. Graph Creation
  8. Model Architectures
  9. Linear Baselines
  10. Text Features with GNN Training
  11. Text Embeddings Fine-Tuning
  12. Text Embeddings Fine-Tuning with GNN Training
  13. Experiments Setup
  14. Results
  15. Limitations
  16. ...and 2 more sections

Figures (1)

  • Figure 1: An example of a subgraph. Here the blue color is the label "Experiences", the red color is the multilabel "Experiences" and "Characteristics".