Talk to Me, Not the Slides: A Real-Time Wearable Assistant for Improving Eye Contact in Presentations

TL;DR

SpeakAssis presents a real-time wearable system that uses a head-mounted eye tracker and an anchor-face tracking approach to provide discreet audio eye-contact guidance during live presentations. By registering the audience in advance and continuously tracking gaze distribution across audience members and non-audience regions, the system detects insufficient or imbalanced eye contact and delivers context-aware prompts via an earphone. In a user study with 4 speakers and 24 audience members, SpeakAssis increased eye-contact duration by 62.5% and improved gaze distribution entropy by 17.4%, while audience members reported higher engagement for sessions employing SpeakAssis. The work also introduces a robust, low-latency anchor-based face identification method and discusses future non-intrusive feedback modalities to further enhance on-stage eye contact and presenter performance.

Abstract

Effective eye contact is a cornerstone of successful public speaking. It strengthens the speaker's credibility and fosters audience engagement. Yet, managing effective eye contact is a skill that demands extensive training and practice, often posing a significant challenge for novice speakers. In this paper, we present SpeakAssis, the first real-time, in-situ wearable system designed to actively assist speakers in maintaining effective eye contact during live presentations. Leveraging a head-mounted eye tracker for gaze and scene view capture, SpeakAssis continuously monitors and analyzes the speaker's gaze distribution across audience and non-audience regions. When ineffective eye-contact patterns are detected, such as insufficient eye contact, or neglect of certain audience segments, SpeakAssis provides timely, context-aware audio prompts via an earphone to guide the speaker's gaze behavior. We evaluate SpeakAssis through a user study involving eight speakers and 24 audience members. Quantitative results show that SpeakAssis increases speakers' eye-contact duration by 62.5% on average and promotes a more balanced distribution of visual attention. Additionally, statistical analysis based on audience surveys reveals that improvements in speaker's eye-contact behavior significantly enhance the audience's perceived engagement and interactivity during presentations.

Figures (11)

• Figure 1: Hardware platform for SpeakAssis. The speaker wears an eye tracker and an earphone, both connected to the laptop. The eye tracker has a front-facing scene camera and two near-eye cameras.
• Figure 2: The workflow of SpeakAssis. Before the presentation, SpeakAssis performs audience registration. Specifically, the speaker first captures the audience's attention, then uses the eye tracker's scene camera to scan and record all audience members. After that, SpeakAssis detects faces in the video and assigns a facial template and a unique identifier to each audience member. In the presentation stage, SpeakAssis continuously tracks the speaker's gaze distribution across the audience members and non-audience regions. By analyzing this gaze distribution in real time, SpeakAssis detects ineffective eye-contact patterns, generates appropriate eye-contact feedback, and discreetly delivers it to the speaker.
• Figure 3: An illustration of the proposed face identification method in the scene video. We introduce the concept of anchor face, which is a face that can be confidently identified either by the face identification model or by a face tracking algorithm. As an example, in the first frame, we apply the face identification model to all detected faces. The rightmost face, which receives the highest confidence score of $0.92$ (on a scale from $0$ to $1$), is selected as the anchor face (highlighted with a green bounding box). Once identified, the anchor face is tracked across subsequent frames (e.g., the second and third frames) until tracking fails --- for instance, when the anchor face disappears from the scene view in the last frame. Now, as long as the anchor face can be successfully tracked or identified, we can infer the identity of the target face by its relative spatial position to the anchor face. For example, in the first frame, the anchor face is identified as $S_6$. Since the target face lies three positions to the left of the anchor face, it is inferred as $S_3$. If tracking of the anchor face fails or it is no longer visible, we re-execute the face recognition model to select a new anchor face on the current scene frame and continue the process.
• Figure 4: The panorama built from the scene images captured by the eye tracker and the facial templates with the unique identifiers registered for all the audience members by their relative positions from left to right.
• Figure 5: Illustration of face identification examples in real-world scenarios. The top image shows a frame where SpeakAssis successfully selected the anchor face. In the bottom (consecutive) frame, SpeakAssis fails to track the anchor face due to significant face position shifts and is unable to select a new anchor face because of motion blur, particularly illustrated in the top-right region.