Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging LLMs to Create a Haptic Devices' Recommendation System

Yang Liu, Haiwei Dong, Abdulmotaleb El Saddik

TL;DR

Fragmentation of haptic device design knowledge hinders development; this work introduces an LLM-driven haptic agent focused on Grounded Force Feedback (GFF) device recommendations. It combines a creator-centric data pipeline that automates taxonomy extraction and population to 114 entries with an end-user retrieval-driven interface that uses dynamic RAG to deliver context-aware suggestions. The system enforces a dual-search strategy (conditional and semantic) and ranks candidates with a transparent score, presenting shortlists to users and linking to source materials. Experimental evaluation with novices and experts shows superior usability and satisfaction (top-10% UEQ) compared to baseline tools, indicating practical impact for accelerating haptic prototyping and knowledge reuse.

Abstract

Haptic technology has seen significant growth, yet a lack of awareness of existing haptic device design knowledge hinders development. This paper addresses these limitations by leveraging advancements in Large Language Models (LLMs) to develop a haptic agent, focusing specifically on Grounded Force Feedback (GFF) devices recommendation. Our approach involves automating the creation of a structured haptic device database using information from research papers and product specifications. This database enables the recommendation of relevant GFF devices based on user queries. To ensure precise and contextually relevant recommendations, the system employs a dynamic retrieval method that combines both conditional and semantic searches. Benchmarking against the established UEQ and existing haptic device searching tools, the proposed haptic recommendation agent ranks in the top 10\% across all UEQ categories with mean differences favoring the agent in nearly all subscales, and maintains no significant performance bias across different user groups, showcasing superior usability and user satisfaction.

Leveraging LLMs to Create a Haptic Devices' Recommendation System

TL;DR

Fragmentation of haptic device design knowledge hinders development; this work introduces an LLM-driven haptic agent focused on Grounded Force Feedback (GFF) device recommendations. It combines a creator-centric data pipeline that automates taxonomy extraction and population to 114 entries with an end-user retrieval-driven interface that uses dynamic RAG to deliver context-aware suggestions. The system enforces a dual-search strategy (conditional and semantic) and ranks candidates with a transparent score, presenting shortlists to users and linking to source materials. Experimental evaluation with novices and experts shows superior usability and satisfaction (top-10% UEQ) compared to baseline tools, indicating practical impact for accelerating haptic prototyping and knowledge reuse.

Abstract

Haptic technology has seen significant growth, yet a lack of awareness of existing haptic device design knowledge hinders development. This paper addresses these limitations by leveraging advancements in Large Language Models (LLMs) to develop a haptic agent, focusing specifically on Grounded Force Feedback (GFF) devices recommendation. Our approach involves automating the creation of a structured haptic device database using information from research papers and product specifications. This database enables the recommendation of relevant GFF devices based on user queries. To ensure precise and contextually relevant recommendations, the system employs a dynamic retrieval method that combines both conditional and semantic searches. Benchmarking against the established UEQ and existing haptic device searching tools, the proposed haptic recommendation agent ranks in the top 10\% across all UEQ categories with mean differences favoring the agent in nearly all subscales, and maintains no significant performance bias across different user groups, showcasing superior usability and user satisfaction.
Paper Structure (19 sections, 1 equation, 4 figures)

This paper contains 19 sections, 1 equation, 4 figures.

Table of Contents

  1. Introduction
  2. Construction of A Haptic Agent
  3. Creator-Centric System
  4. Haptic Device Sourcing
  5. Taxonomy Construction
  6. Human Review and Database Population
  7. End-User System
  8. Summarizing Prompts and Routing
  9. Candidate Selection
  10. Reranking the Qualified Haptic Devices
  11. Haptic Agent Response Generation
  12. Experiments And Analysis
  13. Experiment Setup
  14. Experimental Procedure
  15. Experimental Results and Analysis
  16. ...and 4 more sections

Figures (4)

  • Figure 1: An snapshot for LLM based Haptic device recommendation system, which mainly composed by 3 aspects: 1) User Prompt: The user input prompt to inquiry the potential haptic devices; 2) Device Recommendation: The LLM responds to the user prompt, which ismainly composed by the readable device recommendation and relative detail device info with link. 3) User input box and sample queries.
  • Figure 2: The pipeline of the creator-centric system of haptic agent to convert haptic device information from raw source materials into a structured database. Steps a)-d) represent the stages within the pipeline, with detailed descriptions available in Section \ref{['sec:creator']}1-3) respectively.
  • Figure 3: The pipeline of end-user system of haptic agent to interpret user queries and recommend relevant haptic devices based on the constructed knowledge base. Steps a)-d) represent the stages within the pipeline, with detailed descriptions available in Section \ref{['sec:user']}1-4) respectively.
  • Figure 4: Overview of Experimental Setups and User Experience Analysis. a) Composition of experiment participants, with creators(2) marked in purple and users(11) in blue; b) Distribution of participants by years of experience with haptic devices; c) Density distribution of UEQ across six subscales, with responses ranging from -3 (strongly disagree) to +3 (strongly agree) - responses below 0 are not shown as there were no answers in this range.