Continual Human-in-the-Loop Optimization

TL;DR

The paper tackles the problem that optimal input settings for interactive VR/AR systems vary across users and that traditional HiLO approaches are inefficient for onboarding new users. It formalizes Continual Human-in-the-Loop Optimization (CHiLO) and introduces ConBO, a population-informed Bayesian optimization framework that uses a Bayesian Neural Network as a population surrogate together with user-specific Gaussian Processes, augmented by a memory-replay mechanism to prevent forgetting. ConBO achieves faster, more robust adaptation by leveraging accumulated knowledge from prior users, demonstrated both in simulated benchmark tests (e.g., Branin, McCormick) and a real-world mid-air keyboard personalization study in VR, where it outperforms standard Bayesian optimization and manual tuning in terms of learning speed and final performance. The work provides a principled pathway for scalable, continually improving personalization of HCI systems, with clear directions for extending CHiLO to multi-objective tasks, cross-application transfer, and real-world deployment. Equations such as $x_u^* = \arg\max_{\bm{x} \in \mathcal{X}} f_u(\bm{x})$ and the probabilistic surrogate relation $q(f(\bm{x}) | \bm{x}, \mathcal{D}) = \mathcal{N}(\mu(\bm{x}), \sigma^2(\bm{x}))$ illustrate the formal grounding of the optimization and uncertainty modeling employed by CHiLO and ConBO.

Abstract

Optimal input settings vary across users due to differences in motor abilities and personal preferences, which are typically addressed by manual tuning or calibration. Although human-in-the-loop optimization has the potential to identify optimal settings during use, it is rarely applied due to its long optimization process. A more efficient approach would continually leverage data from previous users to accelerate optimization, exploiting shared traits while adapting to individual characteristics. We introduce the concept of Continual Human-in-the-Loop Optimization and a Bayesian optimization-based method that leverages a Bayesian-neural-network surrogate model to capture population-level characteristics while adapting to new users. We propose a generative replay strategy to mitigate catastrophic forgetting. We demonstrate our method by optimizing virtual reality keyboard parameters for text entry using direct touch, showing reduced adaptation times with a growing user base. Our method opens the door for next-generation personalized input systems that improve with accumulated experience.

Figures (8)

• Figure 1: Illustration of Continual Human-in-the-Loop Optimization (CHiLO): The optimizer, parameterized by $\theta$, continuously evolves by accumulating experience from optimizations for different users. This enables more efficient adaptation to new users over time.
• Figure 2: Illustration of the key elements and workflow of ConBO: ConBO utilizes a population model (BNN) to continually learn the population-level user characteristics. It is further trained by previous user models ($GP_{1,2,3,4...}$). We sample points across the design space and query the predicted means and variances from all GPs (meta-training). ConBO filters out unreliable predictions, and trains the population model with the rest reliable predictions (variance filter). When deploying on a new user $u$, the population model and a user-specific model ($GP_u$) jointly guide the optimization process (adapting to new users). When an adaptation is complete, this user's model will be stored as a previous model.
• Figure 3: Results of our user study. (a) The net WPM at each iteration for three adaptation procedures; the $*$ sign indicates a significant difference found between the ConBO and at that iteration. The scattered dots visualize each data point from individual participants. (b) The regret across all the iterations for all three procedures; the $*$ sign indicates a significant difference between the ConBO and Ṫhe scattered dots visualize each data point from individual participants. (c) The average regret for all conditions; a significant difference found ConBO and (̇d) The mean regret values for each user group using ; the $*$ sign indicates a significant difference between the groups. (e) The mean regret values for each user group using . (f) The mean regret values for each user group using .
• Figure 4: Results of our simulation study using the Branin function (Test 1). All error bars represent one standard deviation. (a) The regret values over the iterations of different BNN-based methods. We highlight that our ConBO without GP has better performance over other BNN-based population models. (b) The regret values over the iterations of GP-based methods, highlighting ConBO without GP outperforms Single GP and delivers comparable performance to TAF. (c) The regret values over the iterations to compare ConBO with and without integrating the user-specific model. The result shows that, with the support of the user-specific model, ConBO can improve its performance further. (d) The mean computation time spent in one iteration for each user. The result highlights the computation costs of GP-based population models (Single GP and TAF) increase quickly when accumulating more data.
• Figure 5: The predicted variance values derived from different BNN-based population model implementations after all 15 user functions. Our ConBO utilizes previous GPs to generate predicted means and variances and further utilizes a variance filter to remove unreliable predictions. Therefore, ConBO allows for more regulated variance predictions. On the other hand, other BNN-based approaches do not have an explicit mechanism to regulate the variance of the population model, making it unstable and changing drastically over the parameter space when trained with a large number of observations. This highlights the stability offered by our approach.