Cost-Aware Bayesian Optimization for Prototyping Interactive Devices

TL;DR

The key idea is to use designer-estimated costs to guide sampling toward more cost-effective prototypes, and results show that accounting for prototyping costs can make Bayesian optimization more compatible with real-world design projects.

Abstract

Deciding which idea is worth prototyping is a central concern in iterative design. A prototype should be produced when the expected improvement is high and the cost is low. However, this is hard to decide, because costs can vary drastically: a simple parameter tweak may take seconds, while fabricating hardware consumes material and energy. Such asymmetries, can discourage a designer from exploring the design space. In this paper, we present an extension of cost-aware Bayesian optimization to account for diverse prototyping costs. The method builds on the power of Bayesian optimization and requires only a minimal modification to the acquisition function. The key idea is to use designer-estimated costs to guide sampling toward more cost-effective prototypes. In technical evaluations, the method achieved comparable utility to a cost-agnostic baseline while requiring only ${\approx}70\%$ of the cost; under strict budgets, it outperformed the baseline threefold. A within-subjects study with 12 participants in a realistic joystick design task demonstrated similar benefits. These results show that accounting for prototyping costs can make Bayesian optimization more compatible with real-world design projects.

Figures (19)

• Figure 1: Top: In standard Bayesian optimization, a surrogate model (e.g., a Gaussian Process) predicts utility, and the expected improvement (EI) directly defines the acquisition function. This approach is cost-blind; it treats all evaluations as equally expensive. Bottom: Cost-aware Bayesian optimization incorporates a cost model by dividing EI by the estimated cost $c(\mathbf{x})$, yielding expected improvement per unit costlee2020cost . This shifts the focus from being purely sample-efficient to being cost-efficient, prioritizing low-cost, high-improvement evaluations. Colors correspond to a tweak ($), swap ($$), or create ($$$) parameter .
• Figure 2: A classification for costs in iterative prototyping that accounts for already-produced prototypes.tweak components are unchanged since the last iteration and incur little-to-no cost. swap components have been used in a previous configuration and can be reused at moderate cost. create components have not been built before and must be fabricated or implemented from scratch, incurring the highest cost. Designers fill in these cost types for the cost-aware Bayesian optimization.
• Figure 3: A snapshot of the iterative optimization process at a single timestep. Our method continuously cycles through sampling, testing, and updating the model; here, we visualize the components used to select the next sample based on the history of previous samples (red dots). $x_1$ and $x_2$ represent prototype parameters. Left: The Rosenbrock ground truth. Center Left: The Gaussian Process (GP) surrogate model approximating the truth based on current samples. Center: Expected Improvement (EI) derived from the GP. Center Right: The cost landscape for $x_1$ and $x_2$; previously built configurations create low-cost vertical and horizontal bands (biasing the search toward tweak and swap operations). Right: The final cost-aware acquisition function, which scales EI by the cost model. The tolerance $\sigma_g$ has been set high to improve visibility.
• Figure 4: Study 1. Cost-aware vs. baseline BO with fixed iterations. A: Performance metrics. Left: Baseline converges to slightly lower regret. Center: Cost-aware achieves similar utility at $\sim$2/3 the cost. Vertical lines mark the cost-aware minimum. Right: Baseline accumulates cost significantly faster. (Means exclude runs with $<95\%$ trials). B: Sample counts per hardware ($x_1$) and software ($x_2$) operation. Baseline over-samples expensive create edits, while cost-aware distributes samples evenly across cost structures.
• Figure 5: Study 2.Tp[]: Final regret and regret distributions across fixed cost budgets ranging from 600 to 7000 units. Others: Cross-section at specific budgets, corresponding to the vertical lines in the left plot. The cost-aware approach significantly outperforms the baseline in the middle segment, where the total budget is not negligible but also not inconsequentially big.