AdaptAI: A Personalized Solution to Sense Your Stress, Fix Your Mess, and Boost Productivity

TL;DR

AdaptAI tackles the challenge of personalized productivity and well-being support by fusing multimodal sensing (egocentric vision, audio, heart activity, and motion) with an agentic LLM workflow. It introduces a Processing Module to convert multimodal data into actionable representations, Routine Table generation to track micro-activities, External Task Agents for lightweight automation, and a Tone-adaptive Conversation Agent to tailor interactions to the user's psychological state. In a preliminary within-subject study with 15 professionals across diverse tasks, AdaptAI reduced cognitive and physical workload and improved focus and satisfaction, while providing transparent reasoning for interventions. The work demonstrates a path toward real-time, personalized workplace assistants, though it also highlights privacy considerations and the need for longer-term, real-world evaluations and privacy-preserving mechanisms.

Abstract

Personalization is a critical yet often overlooked factor in boosting productivity and wellbeing in knowledge-intensive workplaces to better address individual preferences. Existing tools typically offer uniform guidance whether auto-generating email responses or prompting break reminders without accounting for individual behavioral patterns or stress triggers. We introduce AdaptAI, a multimodal AI solution combining egocentric vision and audio, heart and motion activities, and the agentic workflow of Large Language Models LLMs to deliver highly personalized productivity support and context-aware well-being interventions. AdaptAI not only automates peripheral tasks (e.g. drafting succinct document summaries, replying to emails etc.) but also continuously monitors the users unique physiological and situational indicators to dynamically tailor interventions such as micro-break suggestions or exercise prompts, at the exact point of need. In a preliminary study with 15 participants, AdaptAI demonstrated significant improvements in task throughput and user satisfaction by anticipating user stressors and streamlining daily workflows.

Figures (6)

• Figure 1: AdaptAI's Architecture. (1) Processing Module integrates real-time streams from vision, audio, motion, and heart activity data, creating LLM-compatible representations; (2) External Task Agents; (3) Personalized Well-being Intervention Pipeline leverages vision insights for impactful physiological interventions; (4) Tone-Adaptive Conversation Agent (TCA) dynamically adjusts tone using heart activity data to address psychological states and support task queries.
• Figure 2: Example of a User's Daily Routine. The figure illustrates AdaptAI's detection of activity descriptions, classes, surrounding context, and criticality.
• Figure 3: AdaptAI displays real-time Intervention, while correctly accessing the conditions and extracting insights.
• Figure 4: Participants’ ratings on NASA-TLX questions (scale: 1-low to 7-high) for the 4 tasks in control and AdaptAI groups. All highlighted (in gray) are rating differences that are statistically significant with p < 0.05 via Wilcoxon signed-rank tests.
• Figure 5: Improved VLM Captioning Performance Example