Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Man-Made Heuristics Are Dead. Long Live Code Generators!

Rohit Dwivedula, Divyanshu Saxena, Aditya Akella, Swarat Chaudhuri, Daehyeok Kim

TL;DR

The paper tackles the brittleness of hand-crafted system policies by reframing policy design as an automated search problem guided by LLM-generated code. PolicySmith combines a Template-Evaluator-Checker pipeline with evolutionary search to synthesize instance-optimal policies tailored to specific workloads and hardware, demonstrated on web caching and Linux kernel congestion control. Empirical results show PolicySmith-generated cache heuristics outperform multiple baselines on real traces, and kernel-space experiments illustrate feasibility despite safety constraints. The work points to a practical path toward self-evolving, context-aware system policies while outlining open challenges in context drift detection, cross-policy coordination, and offline evaluation.

Abstract

Policy design for various systems controllers has conventionally been a manual process, with domain experts carefully tailoring heuristics for the specific instance in which the policy will be deployed. In this paper, we re-imagine policy design via a novel automated search technique fueled by recent advances in generative models, specifically Large Language Model (LLM)-driven code generation. We outline the design and implementation of PolicySmith, a framework that applies LLMs to synthesize instance-optimal heuristics. We apply PolicySmith to two long-standing systems policies - web caching and congestion control, highlighting the opportunities unraveled by this LLM-driven heuristic search. For caching, PolicySmith discovers heuristics that outperform established baselines on standard open-source traces. For congestion control, we show that PolicySmith can generate safe policies that integrate directly into the Linux kernel.

Man-Made Heuristics Are Dead. Long Live Code Generators!

TL;DR

The paper tackles the brittleness of hand-crafted system policies by reframing policy design as an automated search problem guided by LLM-generated code. PolicySmith combines a Template-Evaluator-Checker pipeline with evolutionary search to synthesize instance-optimal policies tailored to specific workloads and hardware, demonstrated on web caching and Linux kernel congestion control. Empirical results show PolicySmith-generated cache heuristics outperform multiple baselines on real traces, and kernel-space experiments illustrate feasibility despite safety constraints. The work points to a practical path toward self-evolving, context-aware system policies while outlining open challenges in context drift detection, cross-policy coordination, and offline evaluation.

Abstract

Policy design for various systems controllers has conventionally been a manual process, with domain experts carefully tailoring heuristics for the specific instance in which the policy will be deployed. In this paper, we re-imagine policy design via a novel automated search technique fueled by recent advances in generative models, specifically Large Language Model (LLM)-driven code generation. We outline the design and implementation of PolicySmith, a framework that applies LLMs to synthesize instance-optimal heuristics. We apply PolicySmith to two long-standing systems policies - web caching and congestion control, highlighting the opportunities unraveled by this LLM-driven heuristic search. For caching, PolicySmith discovers heuristics that outperform established baselines on standard open-source traces. For congestion control, we show that PolicySmith can generate safe policies that integrate directly into the Linux kernel.

Paper Structure

This paper contains 26 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Heuristic Design: Status Quo and Vision
  3. PolicySmith
  4. Responding to Context Shifts
  5. Explicit context shifts.
  6. Implicit context shifts.
  7. Case Study: Web Caching
  8. Design
  9. Tradeoffs in template design.
  10. Template definition.
  11. Generator.
  12. Context-specific evaluator.
  13. Results
  14. Methodology.
  15. Baselines and Metrics.
  16. ...and 11 more sections

Figures (2)

  • Figure 1: PolicySmith overview
  • Figure 2: Miss ratio improvements over FIFO for all traces in a dataset (higher is better, triangles indicate mean, heuristics ordered left to right by increasing average). Top five performing baselines shown in plot for brevity.