Personalized Image Generation via Human-in-the-loop Bayesian Optimization

TL;DR

This work tackles the last-mile challenge of aligning diffusion-generated images with a user’s imagined target $x^*$ when language prompts saturate. It introduces MultiBO, a training-free human-in-the-loop framework that uses multi-choice preferential Bayesian optimization to edit self-attention via warp parameters in a diffusion model, operating in the low-dimensional space of $Q$, $K$, and $V$ with warping transformations. With a budget of $B=50$ preference queries, MultiBO outperforms baselines on target alignment metrics (e.g., CLIP-I2I, LPIPS) and is robust to reward-hacking compared to reward-model-based approaches, while requiring no offline training data. These results demonstrate a practical, interactive path to highly personalized image generation, enabling users to refine outputs through rich, multi-choice feedback.

Abstract

Imagine Alice has a specific image $x^\ast$ in her mind, say, the view of the street in which she grew up during her childhood. To generate that exact image, she guides a generative model with multiple rounds of prompting and arrives at an image $x^{p*}$. Although $x^{p*}$ is reasonably close to $x^\ast$, Alice finds it difficult to close that gap using language prompts. This paper aims to narrow this gap by observing that even after language has reached its limits, humans can still tell when a new image $x^+$ is closer to $x^\ast$ than $x^{p*}$. Leveraging this observation, we develop MultiBO (Multi-Choice Preferential Bayesian Optimization) that carefully generates $K$ new images as a function of $x^{p*}$, gets preferential feedback from the user, uses the feedback to guide the diffusion model, and ultimately generates a new set of $K$ images. We show that within $B$ rounds of user feedback, it is possible to arrive much closer to $x^\ast$, even though the generative model has no information about $x^\ast$. Qualitative scores from $30$ users, combined with quantitative metrics compared across $5$ baselines, show promising results, suggesting that multi-choice feedback from humans can be effectively harnessed for personalized image generation.

Figures (17)

• Figure 1: The flow of ideas in MultiBO: The BO optimization presents the user with $K$ images and the user chooses $N$ out of $K$ images based on closeness to user's imagined image $x^*$. BO accepts the $N$-out-of-$K$ preferential user feedback and optimizes on the space of transformations applied to the self-attention $KQV$ features, to generate next round of $K$ images, iteratively moving closer to $x^*$.
• Figure 2: MultiBO Image Personalization Pipeline. MultiBO optimizes the self attention $Q,K,V$ features at time interval $[t_{\text{BO}}, t_{\text{BO}} + \Delta]$ over warping transformation space $\mathcal{Y}$. At each iteration $i$, MultiBO offers $K_i$ transform parameter choices and the user picks the $N$ "best" option(s) from the corresponding $K$ attention-modified images. The MultiwiseGPR likelihood models the unobservable user satisfaction function $f$ from the user preferences and the Dynamic Balanced Subspace (DBS) acquisition function prescribes the next set of $K_{i+1}$ warp parameters.
• Figure 3: Qualitative Results-- MultiBO optimization progress: Starting image $x_0 = D_{\theta}(x_t)$ after prompting, best image $\hat{x}^*$ after $B=10,20,30,50$ iters, and the true target $x^*$. For prompts : "A person in a suit holding a sword.", "A forest with blue flowers illustrated in a digital matte style by Dan Mumford and M.W Kaluta.", "A swirling, multicolored portal emerges from the depths of an ocean of coffee, with waves of the rich liquid gently rippling outward. The portal engulfs a coffee cup, which serves as a gateway to a fantastical dimension. The surrounding digital art landscape reflects the colors of the portal, creating an alluring scene of endless possibilities.","an electron cloud model is displayed in vibrant colors with a light spectrum background, showcasing the probability distribution of electrons around the nucleus. the image resembles digital art with pixelated elements, bringing a modern, educational twist to atomic structure visualization.", "The fragrant flowers bloomed on the sturdy stem and the thorny bush.".
• Figure 4: Qualitative comparison of MultiBO ($B=50$), MultiBO$_{\text{Aesthetic}}$, DNO$_{\text{Aesthetic}}$, DEMON$_{\text{Aesthetic}}$, and DNO$_{\text{Aesthetic}}$. For prompts: "A vividly realistic depiction of a snowy Swedish lake at night with hyper-detailed, cinematic-level artistry showcased on ArtStation.", "On the rooftop of a skyscraper in a bustling cyberpunk city, a figure in a trench coat and neon-lit visor stands amidst a garden of bio-luminescent plants, overlooking the maze of flying cars and towering holograms. Robotic birds flit among the foliage, digital billboards flash advertisements in the distance.", "A cyberpunk woman on a motorbike drives away down a street while wearing sunglasses."
• Figure 5: Qualitative results comparing MultiBO (1st & 3rd row) and DEMON choose generate(2nd & 4th row). For prompts: "A wolf wearing a sheep halloween costume going trick-or-treating at the farm", "a blue balloon and a orange bench".