Human-in-the-Loop Optimization with Model-Informed Priors

TL;DR

This work tackles the cold-start challenge in human-in-the-loop optimization by pretraining optimizers on large-scale synthetic user data generated from predictive models. It introduces HOMI, a framework that uses meta-BO trained with synthetic users to learn robust adaptation strategies before real-user deployment, and presents Neural Acquisition Function$^+$ (NAF$^+$), a neural, meta-learned acquisition function augmented with a novelty detector for out-of-distribution robustness and dynamic multi-objective weighting. Through synthetic tests and a real user study on mid-air keyboard adaptation, the approach demonstrates faster early convergence and robust performance across diverse user profiles compared with standard BO, TAF, and ConBO baselines. The work suggests a scalable path toward personalized, efficient interface optimization by combining model-informed priors with in situ optimization, potentially generalizable to a wide range of interactive systems. Overall, HOMI offers a principled method to leverage synthetic user models as training resources, enabling rapid, adaptive interface personalization in VR and beyond.

Abstract

Human-in-the-loop optimization identifies optimal interface designs by iteratively observing user performance. However, it often requires numerous iterations due to the lack of prior information. While recent approaches have accelerated this process by leveraging previous optimization data, collecting user data remains costly and often impractical. We present a conceptual framework, Human-in-the-Loop Optimization with Model-Informed Priors (HOMI), which augments human-in-the-loop optimization with a training phase where the optimizer learns adaptation strategies from diverse, synthetic user data generated with predictive models before deployment. To realize HOMI, we introduce Neural Acquisition Function+ (NAF+), a Bayesian optimization method featuring a neural acquisition function trained with reinforcement learning. NAF+ learns optimization strategies from large-scale synthetic data, improving efficiency in real-time optimization with users. We evaluate HOMI and NAF+ with mid-air keyboard optimization, a representative VR input task. Our work presents a new approach for more efficient interface adaptation by bridging in situ and in silico optimization processes.

Figures (7)

• Figure 1: The general steps of our HOMI framework. Step 1: Model selection Designers or developers select a model, parameterized by $\theta$, that are relevant to the target task, and optionally fit the model's parameters (e.g., Fitts’ law, with parameters $a$ and $b$) using minimal user data to better reflect the target context. Step 2: Synthetic user generation — A diverse set of synthetic users is created by sampling different parameter settings ${\theta_1, \theta_2, \theta_3, \dots}$ from the fitted model's parameter distribution. Step 3: Meta-BO training — The meta-optimizer interacts extensively with these synthetic users to learn efficient strategies for optimizing across user variability. Step 4: Deployment — The trained meta-BO is deployed with real users and quickly adapts to individual performance or preferences using the learned prior experience.
• Figure 2: Our Neural Acquisition Function$^+$ (NAF$^+$) has four main components. Here, we illustrate how they work together using keyboard adaptation as an example case. The surrogate model (Gaussian Process regression) captures the properties of the target user. The information of GP will then allow the neural acquisition function (pre-trained by synthetic data) and the typical acquisition function to generate a set of acquisition values. A novelty detector estimates how likely the new user is different from our training dataset based on the current observations; this information is then used to condition the aggregation of two acquisition functions. Finally, NAF$^+$ suggests a design that is most likely to yield optimal user performance.
• Figure 3: The weighted-sum performance achieved by different conditions (optimization methods) at each iteration. This plot presents the running best performance (i.e., the best performance of each participant achieved up to this iteration), which is a common way of analyzing iterative optimization tasks. The * sign indicates significant differences ($p<0.05$). The thicker, black line in each box shows the mean value, and the thin, orange line indicates the median value.
• Figure 4: The typing speed at each iteration, which is then contributed to the calculation of the final objective function as described in \ref{['sec:objective_function']}. The thicker, black line in each box shows the mean value, and the thin, orange line indicates the median value.
• Figure 5: The error rate at each iteration, which is then contributed to the calculation of the final objective function as described in \ref{['sec:objective_function']}. The thicker, black line in each box shows the mean value, and the thin, orange line indicates the median value.