Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System

Ashish Bastola, Hao Wang, Judsen Hembree, Pooja Yadav, Zihao Gong, Emma Dixon, Abolfazl Razi, Nathan McNeese

TL;DR

This paper presents an LLM-based Smart Reply (LSR) system utilizing the ChatGPT to generate personalized responses in professional collaborative scenarios while adapting to context and communication style based on prior responses and indicates that the proposed LSR reduces overall workload, as measured by the NASA TLX, and improves work performance and productivity in the N-back task.

Abstract

Interactive user interfaces have increasingly explored AI's role in enhancing communication efficiency and productivity in collaborative tasks. The emergence of Large Language Models (LLMs) such as ChatGPT has revolutionized conversational agents, employing advanced deep learning techniques to generate context-aware, coherent, and personalized responses. Consequently, LLM-based AI assistants provide a more natural and efficient user experience across various scenarios. In this paper, we study how LLM models can be used to improve work efficiency in collaborative workplaces. Specifically, we present an LLM-based Smart Reply (LSR) system utilizing the ChatGPT to generate personalized responses in professional collaborative scenarios while adapting to context and communication style based on prior responses. Our two-step process involves generating a preliminary response type (e.g., Agree, Disagree) to provide a generalized direction for message generation, thus reducing response drafting time. We conducted an experiment where participants completed simulated work tasks involving a Dual N-back test and subtask scheduling through Google Calendar while interacting with co-workers. Our findings indicate that the proposed LSR reduces overall workload, as measured by the NASA TLX, and improves work performance and productivity in the N-back task. We also provide qualitative analysis based on participants' experiences, as well as design considerations to provide future directions for improving such implementations.

LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System

TL;DR

This paper presents an LLM-based Smart Reply (LSR) system utilizing the ChatGPT to generate personalized responses in professional collaborative scenarios while adapting to context and communication style based on prior responses and indicates that the proposed LSR reduces overall workload, as measured by the NASA TLX, and improves work performance and productivity in the N-back task.

Abstract

Interactive user interfaces have increasingly explored AI's role in enhancing communication efficiency and productivity in collaborative tasks. The emergence of Large Language Models (LLMs) such as ChatGPT has revolutionized conversational agents, employing advanced deep learning techniques to generate context-aware, coherent, and personalized responses. Consequently, LLM-based AI assistants provide a more natural and efficient user experience across various scenarios. In this paper, we study how LLM models can be used to improve work efficiency in collaborative workplaces. Specifically, we present an LLM-based Smart Reply (LSR) system utilizing the ChatGPT to generate personalized responses in professional collaborative scenarios while adapting to context and communication style based on prior responses. Our two-step process involves generating a preliminary response type (e.g., Agree, Disagree) to provide a generalized direction for message generation, thus reducing response drafting time. We conducted an experiment where participants completed simulated work tasks involving a Dual N-back test and subtask scheduling through Google Calendar while interacting with co-workers. Our findings indicate that the proposed LSR reduces overall workload, as measured by the NASA TLX, and improves work performance and productivity in the N-back task. We also provide qualitative analysis based on participants' experiences, as well as design considerations to provide future directions for improving such implementations.
Paper Structure (47 sections, 8 figures, 2 tables)

This paper contains 47 sections, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. User Interactions with Conventional Smart Reply Systems
  4. Conversational Agents in the Workplace
  5. Method
  6. System Design
  7. Simulated Work via the N-back Test
  8. Subtask Management Using Google Calendar
  9. ChatGPT Integration With Slack
  10. Participants
  11. Data Collection
  12. Work Performance Tracking
  13. Productivity Tracking
  14. Cognitive Effort Tracking
  15. User Experience Tracking
  16. ...and 32 more sections

Figures (8)

  • Figure 1: The overall workflow of experiment
  • Figure 2: The main component of the proposed user interface. (a) N-Back test program; (b) Slack with LSR (top) and Google Calendar (bottom); (c) ChatGPT prompting template.
  • Figure 3: Result of LSR test in collaborative work. (a) Work performance test with LSR; (b) Productivity test with LSR.
  • Figure 4: TLX Comparison
  • Figure 5: Questionnaire for Measuring the General Response to LSR
  • ...and 3 more figures