Touched by ChatGPT: Using an LLM to Drive Affective Tactile Interaction

TL;DR

This work addresses how to convey affective states through touch by leveraging an LLM to generate 10-second vibrotactile patterns mapped onto a $5\times5$ motor sleeve. Using chain prompts with an LLM (gpt-4o), the system translates textual emotion and gesture descriptions into time-evolving, spatially distributed vibration data, which is then played back via a wearable device and decoded by human participants. Across 32 participants, the approach achieved an overall emotion-decoding accuracy of $30.3\%$ (well above the $10\%$ chance level) and gesture-decoding accuracy notably higher for tickle and rub, indicating that LLM-generated tactile data can convey meaningful affective cues, though some expressions (e.g., confusion) remain challenging. The study demonstrates the potential of tactile-language integration to enhance human-robot interaction and suggests future improvements in hardware resolution and repertoire expansion to boost fidelity and differentiation of tactile expressions.

Abstract

Touch is a fundamental aspect of emotion-rich communication, playing a vital role in human interaction and offering significant potential in human-robot interaction. Previous research has demonstrated that a sparse representation of human touch can effectively convey social tactile signals. However, advances in human-robot tactile interaction remain limited, as many humanoid robots possess simplistic capabilities, such as only opening and closing their hands, restricting nuanced tactile expressions. In this study, we explore how a robot can use sparse representations of tactile vibrations to convey emotions to a person. To achieve this, we developed a wearable sleeve integrated with a 5x5 grid of vibration motors, enabling the robot to communicate diverse tactile emotions and gestures. Using chain prompts within a Large Language Model (LLM), we generated distinct 10-second vibration patterns corresponding to 10 emotions (e.g., happiness, sadness, fear) and 6 touch gestures (e.g., pat, rub, tap). Participants (N = 32) then rated each vibration stimulus based on perceived valence and arousal. People are accurate at recognising intended emotions, a result which aligns with earlier findings. These results highlight the LLM's ability to generate emotional haptic data and effectively convey emotions through tactile signals. By translating complex emotional and tactile expressions into vibratory patterns, this research demonstrates how LLMs can enhance physical interaction between humans and robots.

Figures (5)

• Figure 1: Experimental setup.
• Figure 2: Flow chart of LLM generated vibration data.
• Figure 3: Illustration of frames of LLM generated rub gesture.
• Figure 4: Arousal and valence rating distribution of emotions.
• Figure 5: Confusion matrix.