Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SocializeChat: A GPT-Based AAC Tool Grounded in Personal Memories to Support Social Communication

Wei Xiang, Yunkai Xu, Yuyang Fang, Zhuyu Teng, Zhaoqu Jiang, Beijia Hu, Jinguo Yang

TL;DR

SocializeChat targets the gap in AAC tools for social, memory-rich conversations among elderly individuals with speech impairments. By grounding sentence suggestions in users' personal memories and partner personas, and by incorporating topic preferences and interpersonal closeness, the system delivers contextually relevant, relatable dialogue options generated via GPT-4. The approach combines memory retrieval, a six-part prompting framework, and user-controlled customization to balance automation with autonomy. A user study with 16 participants demonstrated meaningful improvements in relevance and engagement, alongside acceptable usability, suggesting potential to reduce social isolation and promote memory-based social interaction in real-world settings. Overall, the work emphasizes memory, identity, and personalization as central design principles for inclusive, AI-assisted communication technologies.

Abstract

Elderly people with speech impairments often face challenges in engaging in meaningful social communication, particularly when using Augmentative and Alternative Communication (AAC) tools that primarily address basic needs. Moreover, effective chats often rely on personal memories, which is hard to extract and reuse. We introduce SocializeChat, an AAC tool that generates sentence suggestions by drawing on users' personal memory records. By incorporating topic preference and interpersonal closeness, the system reuses past experience and tailors suggestions to different social contexts and conversation partners. SocializeChat not only leverages past experiences to support interaction, but also treats conversations as opportunities to create new memories, fostering a dynamic cycle between memory and communication. A user study shows its potential to enhance the inclusivity and relevance of AAC-supported social interaction.

SocializeChat: A GPT-Based AAC Tool Grounded in Personal Memories to Support Social Communication

TL;DR

SocializeChat targets the gap in AAC tools for social, memory-rich conversations among elderly individuals with speech impairments. By grounding sentence suggestions in users' personal memories and partner personas, and by incorporating topic preferences and interpersonal closeness, the system delivers contextually relevant, relatable dialogue options generated via GPT-4. The approach combines memory retrieval, a six-part prompting framework, and user-controlled customization to balance automation with autonomy. A user study with 16 participants demonstrated meaningful improvements in relevance and engagement, alongside acceptable usability, suggesting potential to reduce social isolation and promote memory-based social interaction in real-world settings. Overall, the work emphasizes memory, identity, and personalization as central design principles for inclusive, AI-assisted communication technologies.

Abstract

Elderly people with speech impairments often face challenges in engaging in meaningful social communication, particularly when using Augmentative and Alternative Communication (AAC) tools that primarily address basic needs. Moreover, effective chats often rely on personal memories, which is hard to extract and reuse. We introduce SocializeChat, an AAC tool that generates sentence suggestions by drawing on users' personal memory records. By incorporating topic preference and interpersonal closeness, the system reuses past experience and tailors suggestions to different social contexts and conversation partners. SocializeChat not only leverages past experiences to support interaction, but also treats conversations as opportunities to create new memories, fostering a dynamic cycle between memory and communication. A user study shows its potential to enhance the inclusivity and relevance of AAC-supported social interaction.
Paper Structure (26 sections, 2 figures, 1 table)

This paper contains 26 sections, 2 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Related Works
  3. Social Communication with AAC
  4. Influential Dimensions in Social Communication
  5. Memory in Social Communication
  6. System Construction
  7. User Flow and Interface Design
  8. Sentence Generation Based on User Memory Records
  9. Preprocessing of Current Conversation with User Records
  10. Prompts for Generation
  11. Methods of Customizing Generated Sentences
  12. User Study
  13. Procedure
  14. Step 1: Introduction to SocializeChat (10 min)
  15. Step 2: Trying out the Tool (30 min)
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Usage Flow: When the conversation partner (the person communicating with the user) is speaking to user (A). Voice input is transcribed and combined with retrieved memory records and partner persona as LLM prompt input; adjusted tags are generated alongside the response. User(the AAC user operating SocializeChat) can choose one of the provided sentence suggestions to engage in conversation or start a new conversation (B).
  • Figure 2: Interface Design: The main interface is divided into three sections: (A) system status and user identity ($a_1$, $a_2$); (B) the conversation interaction area, including sentence suggestions tagged with closeness levels ($b_1$), contextual tag adjustments ($b_2$), and a virtual keyboard for manual input ($b_3$); and (C) a navigation menu ($c$). Users can also initiate new topics based on partner preferences via a dedicated button ($d$).