Integrating Physiological Data with Large Language Models for Empathic Human-AI Interaction

TL;DR

This work introduces physiology-driven EmLLMs that integrate wearable-derived biosignals with large language models to produce empathic human-AI interactions. It employs a multichannel 1D CNN to infer psychological states from EDA, BVP, and Skin Temperature data and uses QLoRA to fine-tune Falcon-7B, embedding inferred stress states into empathetic, CBT-informed prompts. A pilot study with 8 students demonstrates the approach's potential, showing competitive stress-state prediction and a meaningful therapeutic alliance, while highlighting challenges in language behavior, memory, and privacy. The results advocate for broader personalization and ethically aligned, privacy-preserving designs to advance digital psychotherapy tools.

Abstract

This paper explores enhancing empathy in Large Language Models (LLMs) by integrating them with physiological data. We propose a physiological computing approach that includes developing deep learning models that use physiological data for recognizing psychological states and integrating the predicted states with LLMs for empathic interaction. We showcase the application of this approach in an Empathic LLM (EmLLM) chatbot for stress monitoring and control. We also discuss the results of a pilot study that evaluates this EmLLM chatbot based on its ability to accurately predict user stress, provide human-like responses, and assess the therapeutic alliance with the user.

Figures (2)

• Figure 1: Proposed EmLLM approach.
• Figure 2: Average stress states of all participants.