Trinity: Synchronizing Verbal, Nonverbal, and Visual Channels to Support Academic Oral Presentation Delivery

TL;DR

This work introduces Trinity, a hybrid mobile-centric delivery support system that provides guidance for multichannel delivery on-the-fly and is perceived as significantly more helpful than baselines, without excessive cognitive load.

Abstract

Academic Oral Presentation (AOP) allows English-As-Foreign-Language (EFL) students to express ideas, engage in academic discourse, and present research findings. However, while previous efforts focus on training efficiency or speech assistance, EFL students often face the challenge of seamlessly integrating verbal, nonverbal, and visual elements into their presentations to avoid coming across as monotonous and unappealing. Based on a need-finding survey, a design study, and an expert interview, we introduce Trinity, a hybrid mobile-centric delivery support system that provides guidance for multichannel delivery on-the-fly. On the desktop side, Trinity facilitates script refinement and offers customizable delivery support based on large language models (LLMs). Based on the desktop configuration, Trinity App enables a remote mobile visual control, multi-level speech pace modulation, and integrated delivery prompts for synchronized delivery. A controlled between-subject user study suggests that Trinity effectively supports AOP delivery and is perceived as significantly more helpful than baselines, without excessive cognitive load.

Figures (23)

• Figure 1: Taxonomy of existing works in presentation domain and coverage of our work.
• Figure 2: EFL student's needs and perceived importance from instructors based on a normalized weighted importance score (from $0$ to $1$, the closer to $1$, the more important). The information without color filling represents the factor that is not implemented in the system. A) The ranking of students’ actual needs in factors influencing AOP delivery. B) The ranking of instructors’ perceived importance levels of delivery factors.
• Figure 3: The design development process in the Formative Study, where the blue boxes are different research activities while the gray boxes are the research entities.
• Figure 4: Flow diagram of design formation and Storyboard No.2. A) The design iteration started from storyboarding for scenario and media, then spanned to sketch out designs for verbal, nonverbal, and visual delivery support, ending up with a design for synchronizing diverse communication channels. B) Storyboard No.2: Offer delivery support on the fly. participants thought this would enhance their performance with reduced preparation effort.
• Figure 5: Designs of previous delivery support tools. A) Office Remote, an Office subsystem that turns phone into a smart remote that interacts with Microsoft Office on PC. Picture adopted from officeRemotePicture. B) Rhema, an intelligent user interface for Google Glass to help people with public speaking. Picture adopted from tanveer2015rhema. C) IntelliPrompter, a speech-based note display system that automatically tracks a presenter’s coverage of each slide’s content. Picture adopted from asadi2017intelliprompter.