From Voice to Value: Leveraging AI to Enhance Spoken Online Reviews on the Go

TL;DR

The paper tackles the challenge of on-the-go review writing by introducing Vocalizer, a voice-based app with LLM-assisted editing to polish user reviews. Through a within-subject, longitudinal field study comparing a voice-only version and an AI-assisted version, it shows that AI-driven refinements increase detail, coherence, willingness to share, and user self-efficacy while raising concerns about authenticity. The work contributes design and evaluation insights, including prompt-design strategies and user-experience implications, highlighting both the practical potential and ethical considerations of AI-assisted review generation. Overall, the findings support integrating AI to enhance user-generated content quality while calling for careful handling of authenticity, trust, and cultural differences in deployment.

Abstract

Online reviews help people make better decisions. Review platforms usually depend on typed input, where leaving a good review requires significant effort because users must carefully organize and articulate their thoughts. This may discourage users from leaving comprehensive and high-quality reviews, especially when they are on the go. To address this challenge, we developed Vocalizer, a mobile application that enables users to provide reviews through voice input, with enhancements from a large language model (LLM). In a longitudinal study, we analysed user interactions with the app, focusing on AI-driven features that help refine and improve reviews. Our findings show that users frequently utilized the AI agent to add more detailed information to their reviews. We also show how interactive AI features can improve users self-efficacy and willingness to share reviews online. Finally, we discuss the opportunities and challenges of integrating AI assistance into review-writing systems.

Figures (8)

• Figure 1: Flowcharts showing the operation of both versions of Vocalizer. On the left, the voice-only version, and on the right, the LLM-assisted version, with the additional LLM-powered processes and user-LLM interactions coloured blue. As shown, the LLM-assisted version offers advice the user can read or ignore, and they can prompt the AI agent to enhance the review if they do not want to edit it themselves.
• Figure 2: Comparison of an example original review and its AI-enhanced revision.
• Figure 3: Core functionalities of the LLM-Assisted Version of Vocalizer presented chronologically from left to right. 1 - Selecting restaurant (A) and recording review (B), 2 - reading transcription and improved version (C), asking AI agent to improve review with a text prompt (D) then reading the newer iteration of the improved review with personalised, AI-generated improvement ideas visible below (E), 3 - entering feedback on sliders and submitting restaurant review (F)
• Figure 4: Study design diagram showing the order of tasks and questionnaires for each participant.
• Figure 5: Feedback response distribution for evaluating the VOV and LAV. The figure illustrates user perceptions regarding the usefulness, overall satisfaction, and willingness to share reviews generated by application versions.