Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Human Feedback into a Reinforcement Learning-Based Framework for Adaptive User Interfaces

Daniel Gaspar-Figueiredo, Marta Fernández-Diego, Silvia Abrahão, Emilio Insfran

TL;DR

This work extends a reinforcement learning-based Adaptive User Interface framework by integrating personalized human feedback (HF) and training individualized RL agents for each user. A dual-reward scheme combines baseline predictive HCI signals with HF-derived modifiers, learned offline via an HF collection process and a per-user preference mechanism. In a two-domain empirical study with 33 participants, adaptive AUIs guided by user-specific HF significantly improved user satisfaction and engagement compared with non-adaptive interfaces. The results support the promise of human-in-the-loop personalization for AUIs and call for standardized, scalable methods to capture and incorporate HF into domain-specific UI adaptations.

Abstract

Adaptive User Interfaces (AUI) play a crucial role in modern software applications by dynamically adjusting interface elements to accommodate users' diverse and evolving needs. However, existing adaptation strategies often lack real-time responsiveness. Reinforcement Learning (RL) has emerged as a promising approach for addressing complex, sequential adaptation challenges, enabling adaptive systems to learn optimal policies based on previous adaptation experiences. Although RL has been applied to AUIs,integrating RL agents effectively within user interactions remains a challenge. In this paper, we enhance a RL-based Adaptive User Interface adaption framework by incorporating personalized human feedback directly into the leaning process. Unlike prior approaches that rely on a single pre-trained RL model, our approach trains a unique RL agent for each user, allowing individuals to actively shape their personal RL agent's policy, potentially leading to more personalized and responsive UI adaptations. To evaluate this approach, we conducted an empirical study to assess the impact of integrating human feedback into the RL-based Adaptive User Interface adaption framework and its effect on User Experience (UX). The study involved 33 participants interacting with AUIs incorporating human feedback and non-adaptive user interfaces in two domains: an e-learning platform and a trip-planning application. The results suggest that incorporating human feedback into RL-driven adaptations significantly enhances UX, offering promising directions for advancing adaptive capabilities and user-centered design in AUIs.

Integrating Human Feedback into a Reinforcement Learning-Based Framework for Adaptive User Interfaces

TL;DR

This work extends a reinforcement learning-based Adaptive User Interface framework by integrating personalized human feedback (HF) and training individualized RL agents for each user. A dual-reward scheme combines baseline predictive HCI signals with HF-derived modifiers, learned offline via an HF collection process and a per-user preference mechanism. In a two-domain empirical study with 33 participants, adaptive AUIs guided by user-specific HF significantly improved user satisfaction and engagement compared with non-adaptive interfaces. The results support the promise of human-in-the-loop personalization for AUIs and call for standardized, scalable methods to capture and incorporate HF into domain-specific UI adaptations.

Abstract

Adaptive User Interfaces (AUI) play a crucial role in modern software applications by dynamically adjusting interface elements to accommodate users' diverse and evolving needs. However, existing adaptation strategies often lack real-time responsiveness. Reinforcement Learning (RL) has emerged as a promising approach for addressing complex, sequential adaptation challenges, enabling adaptive systems to learn optimal policies based on previous adaptation experiences. Although RL has been applied to AUIs,integrating RL agents effectively within user interactions remains a challenge. In this paper, we enhance a RL-based Adaptive User Interface adaption framework by incorporating personalized human feedback directly into the leaning process. Unlike prior approaches that rely on a single pre-trained RL model, our approach trains a unique RL agent for each user, allowing individuals to actively shape their personal RL agent's policy, potentially leading to more personalized and responsive UI adaptations. To evaluate this approach, we conducted an empirical study to assess the impact of integrating human feedback into the RL-based Adaptive User Interface adaption framework and its effect on User Experience (UX). The study involved 33 participants interacting with AUIs incorporating human feedback and non-adaptive user interfaces in two domains: an e-learning platform and a trip-planning application. The results suggest that incorporating human feedback into RL-driven adaptations significantly enhances UX, offering promising directions for advancing adaptive capabilities and user-centered design in AUIs.
Paper Structure (31 sections, 3 figures, 4 tables)

This paper contains 31 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. RL-Based UI Adaptation Framework
  4. Overview of the Original Framework
  5. Extending the RL-based Framework for AUI
  6. Adaptive User Interfaces
  7. Reward Models and Human Feedback for AUIs
  8. Implementation
  9. Experimental Study
  10. Research Question and Hypothesis
  11. Variables
  12. Independent Variables
  13. Dependent Variables
  14. Subjects
  15. Design
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Extended intelligent UI adaptation framework from abrahaoModel:2021. Human Feedback Module and Reward Model are the main contributions in this paper.
  • Figure 2: Experimental methodology: 1) Feedback collection; 2) Individualized RL agent training; and 3) User interaction with AUIs and non-adaptive UIs
  • Figure 3: Violin plots of User Satisfaction and User Engagement by Technique. The blue dashed line represents the neutral midpoint for each scale. The red cross marker indicates the mean value for each technique.