Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LILO: Bayesian Optimization with Interactive Natural Language Feedback

Katarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy

TL;DR

This paper tackles optimizing a black-box objective under a decision maker's nuanced preferences by translating natural language feedback into scalar utilities for Gaussian-process-based Bayesian optimization. The proposed Language-in-the-loop Optimization (LILO) framework places an LLM in a supporting role to interpret feedback and priors while preserving BO's uncertainty quantification through two proxy GPs that model the composite utility and its outcomes. The method iterates through candidate generation, DM-driven feedback elicitation, and utility estimation via LLM labeling guided by EUBO, incorporating prior domain knowledge when available. Empirically, LILO outperforms traditional true-utility and preferential BO baselines, particularly in feedback-scarce regimes, and demonstrates robustness to different LLMs and prompt configurations, highlighting its potential for practical interactive optimization in complex, multi-metric settings.

Abstract

For many real-world applications, feedback is essential in translating complex, nuanced, or subjective goals into quantifiable optimization objectives. We propose a language-in-the-loop framework that uses a large language model (LLM) to convert unstructured feedback in the form of natural language into scalar utilities to conduct BO over a numeric search space. Unlike preferential BO, which only accepts restricted feedback formats and requires customized models for each domain-specific problem, our approach leverages LLMs to turn varied types of textual feedback into consistent utility signals and to easily include flexible user priors without manual kernel design. At the same time, our method maintains the sample efficiency and principled uncertainty quantification of BO. We show that this hybrid method not only provides a more natural interface to the decision maker but also outperforms conventional BO baselines and LLM-only optimizers, particularly in feedback-limited regimes.

LILO: Bayesian Optimization with Interactive Natural Language Feedback

TL;DR

This paper tackles optimizing a black-box objective under a decision maker's nuanced preferences by translating natural language feedback into scalar utilities for Gaussian-process-based Bayesian optimization. The proposed Language-in-the-loop Optimization (LILO) framework places an LLM in a supporting role to interpret feedback and priors while preserving BO's uncertainty quantification through two proxy GPs that model the composite utility and its outcomes. The method iterates through candidate generation, DM-driven feedback elicitation, and utility estimation via LLM labeling guided by EUBO, incorporating prior domain knowledge when available. Empirically, LILO outperforms traditional true-utility and preferential BO baselines, particularly in feedback-scarce regimes, and demonstrates robustness to different LLMs and prompt configurations, highlighting its potential for practical interactive optimization in complex, multi-metric settings.

Abstract

For many real-world applications, feedback is essential in translating complex, nuanced, or subjective goals into quantifiable optimization objectives. We propose a language-in-the-loop framework that uses a large language model (LLM) to convert unstructured feedback in the form of natural language into scalar utilities to conduct BO over a numeric search space. Unlike preferential BO, which only accepts restricted feedback formats and requires customized models for each domain-specific problem, our approach leverages LLMs to turn varied types of textual feedback into consistent utility signals and to easily include flexible user priors without manual kernel design. At the same time, our method maintains the sample efficiency and principled uncertainty quantification of BO. We show that this hybrid method not only provides a more natural interface to the decision maker but also outperforms conventional BO baselines and LLM-only optimizers, particularly in feedback-limited regimes.
Paper Structure (51 sections, 15 equations, 19 figures, 2 tables, 6 algorithms)

This paper contains 51 sections, 15 equations, 19 figures, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Background & Related Works
  4. Preferential Bayesian Optimization.
  5. Optimization with LLMs.
  6. LLM as Few-Shot Learners and Bayesian Predictors.
  7. Method
  8. LILO: Language-in-the-loop Optimization
  9. The entry point.
  10. Candidate Generation & Experimentation.
  11. Feedback Acquisition.
  12. Utility Estimation via LLM Labeling + GP Modeling.
  13. Incorporating Prior Knowledge with LILO
  14. Experiments
  15. Benchmarking Setup
  16. ...and 36 more sections

Figures (19)

  • Figure 1: Core steps of LILO.
  • Figure 2: Overview of LILO: Configurations to be evaluated on the black box function are generated with a BO acquisition function leveraging the GP utility model. After observing associated outcomes, the LLM generates a set of questions for the decision maker based on all previously observed outcomes and user feedback. Responses are added to the LLM's conversation history, based on which the LLM is asked to provide preferential feedback on a set of pairwise comparisons. This feedback is then used to update the GP utility model, after which the iteration continues.
  • Figure 3: Example fragments of feedback elicitation conversation of LILO with the DM in one of the test problem (Thermal Comfort Type A). LILO is able to elicit and incorporate such qualitative feedback about optimization effectively as shown in our experiments in Section \ref{['sec:Experiments']}.
  • Figure 4: Maximum value of the ground-truth utility achieved after $n$ iterations. Error bars are 95% confidence intervals of the mean max value across 30 replications.
  • Figure 5: Comparison of LILO variants. Average results over three DTLZ2 environments (30 replications per environment, values min-max standardized within an environment before aggregation).
  • ...and 14 more figures