Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

See-Through Face Display: Enabling Gaze Communication for Any Face$\unicode{x2013}$Human or AI

Kazuya Izumi, Ryosuke Hyakuta, Ippei Suzuki, Yoichi Ochiai

TL;DR

This paper presents the implementation of See-Through Face Display and examines its potential applications, demonstrating how this compact eye-contact system can enhance gaze communication in both human-to-human and human-AI interactions.

Abstract

We present See-Through Face Display, an eye-contact display system designed to enhance gaze awareness in both human-to-human and human-to-avatar communication. The system addresses the limitations of existing gaze correction methods by combining a transparent display with a strategically positioned camera. The display alternates rapidly between a visible and transparent state, thereby enabling the camera to capture clear images of the user's face from behind the display. This configuration allows for mutual gaze awareness among remote participants without the necessity of a large form factor or computationally resource-intensive image processing. In comparison to conventional methodologies, See-Through Face Display offers a number of practical advantages. The system requires minimal physical space, operates with low computational overhead, and avoids the visual artifacts typically associated with software-based gaze redirection. These features render the system suitable for a variety of applications, including multi-party teleconferencing and remote customer service. Furthermore, the alignment of the camera's field of view with the displayed face position facilitates more natural gaze-based interactions with AI avatars. This paper presents the implementation of See-Through Face Display and examines its potential applications, demonstrating how this compact eye-contact system can enhance gaze communication in both human-to-human and human-AI interactions.

See-Through Face Display: Enabling Gaze Communication for Any Face$\unicode{x2013}$Human or AI

TL;DR

This paper presents the implementation of See-Through Face Display and examines its potential applications, demonstrating how this compact eye-contact system can enhance gaze communication in both human-to-human and human-AI interactions.

Abstract

We present See-Through Face Display, an eye-contact display system designed to enhance gaze awareness in both human-to-human and human-to-avatar communication. The system addresses the limitations of existing gaze correction methods by combining a transparent display with a strategically positioned camera. The display alternates rapidly between a visible and transparent state, thereby enabling the camera to capture clear images of the user's face from behind the display. This configuration allows for mutual gaze awareness among remote participants without the necessity of a large form factor or computationally resource-intensive image processing. In comparison to conventional methodologies, See-Through Face Display offers a number of practical advantages. The system requires minimal physical space, operates with low computational overhead, and avoids the visual artifacts typically associated with software-based gaze redirection. These features render the system suitable for a variety of applications, including multi-party teleconferencing and remote customer service. Furthermore, the alignment of the camera's field of view with the displayed face position facilitates more natural gaze-based interactions with AI avatars. This paper presents the implementation of See-Through Face Display and examines its potential applications, demonstrating how this compact eye-contact system can enhance gaze communication in both human-to-human and human-AI interactions.
Paper Structure (13 sections, 3 figures)

This paper contains 13 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Gaze Awareness in Telepresence
  4. Interactions Using See-Through Displays
  5. System
  6. Hardware Setup
  7. Image Processing
  8. Application Scenarios
  9. Multi-Party Remote Conversations
  10. Remote Reception Services
  11. AI-Assisted Product Recommendations
  12. Discussion
  13. Conclusion

Figures (3)

  • Figure 1: (a) Hardware setup of See-Through Face Display. (b) The transparent display rapidly switches between transparent (None) and scattering (Red, Green, Blue). When the display is transparent, the camera captures the user's face. (c) Image Processing of See-Through Face Display. The captured frames are streamed to a virtual camera using DirectShow Filters and transmitted via the WebRTC protocol.
  • Figure 2: Evaluation of Eye Contact Comparison. (a) A participant watched a picture of a person's face and a nine-part grid of areas near the face as stimulation. (b) Eye contact evaluation when watching the stimulation with a conventional videoconferencing system. (c) Eye contact evaluation when watching the stimulation with See-Through Face Display.
  • Figure 3: An application scenario for AI-assisted product recommendations. When a visitor gazes at the avatar, the avatar talks to him and offers recommendations based on his requests and his current glasses while looking at him.