Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Focus Agent: LLM-Powered Virtual Focus Group

Taiyu Zhang, Xuesong Zhang, Robbe Cools, Adalberto L. Simeone

TL;DR

Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants, and the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.

Abstract

In the domain of Human-Computer Interaction, focus groups represent a widely utilised yet resource-intensive methodology, often demanding the expertise of skilled moderators and meticulous preparatory efforts. This study introduces the ``Focus Agent,'' a Large Language Model (LLM) powered framework that simulates both the focus group (for data collection) and acts as a moderator in a focus group setting with human participants. To assess the data quality derived from the Focus Agent, we ran five focus group sessions with a total of 23 human participants as well as deploying the Focus Agent to simulate these discussions with AI participants. Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants. Furthermore, the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.

Focus Agent: LLM-Powered Virtual Focus Group

TL;DR

Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants, and the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.

Abstract

In the domain of Human-Computer Interaction, focus groups represent a widely utilised yet resource-intensive methodology, often demanding the expertise of skilled moderators and meticulous preparatory efforts. This study introduces the ``Focus Agent,'' a Large Language Model (LLM) powered framework that simulates both the focus group (for data collection) and acts as a moderator in a focus group setting with human participants. To assess the data quality derived from the Focus Agent, we ran five focus group sessions with a total of 23 human participants as well as deploying the Focus Agent to simulate these discussions with AI participants. Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants. Furthermore, the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.
Paper Structure (22 sections, 6 figures, 2 algorithms)

This paper contains 22 sections, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Focus Group Development
  4. Multi-Agent Simulation
  5. Multi-speaker speech recognition for Voice-based Conversational Agents
  6. Focus Agent Implementation
  7. Focus Group Simulation
  8. Voice-based Focus Agent with human participants
  9. Pilot Study
  10. User Study
  11. Result Analysis
  12. Focus Group
  13. Meta Focus Group
  14. Theme 1: User Experiences of Focus Group.
  15. Theme 2: Attitude to the Focus Agent.
  16. ...and 7 more sections

Figures (6)

  • Figure 1: The AI moderator generates questions according to the discussion content and plan, while AI Participants discuss the prompt from the moderator.
  • Figure 2: A web demo of the Focus Group simulation system.
  • Figure 3: Speech to Text system. We divided long audio recording into short pieces with voice activity detection. Then we transcribed the short audio pieces and recognised the speaker according to the voiceprints collected in advance from the participants.
  • Figure 4: Users participant focus group using Focus Agent in VR environment.
  • Figure 5: Content analysis according to the themes from both focus group and focus group simulation, font size indicates the frequency of the codes
  • ...and 1 more figures