Algorithm Generation via Creative Ideation

TL;DR

The paper tackles the challenge of automatic algorithm generation in systems where the solution space is highly discontinuous. It shows that large language models tend to bias toward well-known heuristics, and introduces MetaMuse, a self-reflection framework that uses diversity in feedback space, external stimuli, and waypoint-based execution to enable creative yet executable algorithm ideation. Through two cloud-scale problems, cache replacement and online bin packing, MetaMuse outperforms baselines and human heuristics in both performance and solution diversity, at a low per-solution cost. The work demonstrates a practical path to automated, production-ready algorithm design and reveals surprising, nonintuitive designs that engineers may overlook.

Abstract

Designing system algorithms remains challenging, where the discontinuous nature of the solution space often forces system engineers to rely on generic heuristics at the expense of performance. We study whether LLMs can practically drive algorithm generation, and find that they are biased towards well-known generic designs, rather than making the creative leaps needed to navigate the discontinuous solution space. To address this limitation, we introduce MetaMuse, a framework for creative ideation built on three self-reflection principles: (1) quantifying solution diversity and usefulness in measurable performance space, rather than abstract idea space, (2) steering ideation through external stimuli, rather than internal randomness, and (3) constructing executable solutions using waypoint reasoning, rather than free-form chain-of-thought. Extensive evaluation shows that MetaMuse can generate high-performing solutions for two critical problems at a global cloud provider: cache replacement (reducing cache misses by up to 35.76%) and online bin packing (reducing bin usage by up to 30.93%).

Figures (5)

• Figure 1: Repeatedly sampling LLMs can generate biased solutions. In the cache replacement problem, solutions tend to cluster around well-known ones in literature, e.g., LRU, LFU, and FIFO.
• Figure 2: MetaMuse reflects on previous solutions, in order to think about how to generate subsequent solutions. Each iteration goes through three steps: evaluating the diversity of generated solutions (§\ref{['sec:evaluating_feedback_space']}), steering ideation with external stimuli (§\ref{['sec:steering_stimuli']}), and developing executable solutions (§\ref{['sec:developing_waypoint_reasoning']}).
• Figure 3: Comparisons of top cache solutions generated by MetaMuse and baselines. Box plots show the miss ratio reduction (with respect to FIFO heuristics) that each solution achieves over 96 traces. MetaMuse has higher reduction across nearly all percentiles, on different LLMs.
• Figure 4: Comparisons of top online bin packing solutions generated by MetaMuse and baselines. Box plots show the bin usage reduction (with respect to First Fit heuristics) that each solution achieves over 288 traces. MetaMuse has higher reduction across nearly all percentiles, on different LLMs. We note that some human heuristics are not visibile because they perform worse than First Fit.
• Figure 5: Comparisons of the diversity of cache replacement solutions, generated by different stimuli selection strategies: RSDict and RSDict-SF. "-noWR" means removing waypoint reasoning. We ideate with different LLMs, and code with GPT-4o. RSDict-SF (dashed lines) outperforms RSDict (solid lines). Removing waypoint reasoning (blue lines) reduces solution diversity (red lines).