KAPSO: A Knowledge-grounded framework for Autonomous Program Synthesis and Optimization

TL;DR

KAPSO addresses long-horizon failures in autonomous program synthesis by separating knowledge grounding from execution and tying progress to explicit evaluator outcomes. It combines a git-native experimentation engine, a MediaWiki-based knowledge base with a typed knowledge graph, and a cognitive memory layer that distills lessons from experiment traces into an episodic store, all embedded in a modular, evaluator-driven evolve loop. The framework supports pluggable evaluators and backends and provides a unified deployment interface with multiple strategies, enabling end-to-end optimization across domains. Evaluations on MLE-Bench and ALE-Bench demonstrate end-to-end performance gains and improved efficiency, with reproducible knowledge packages and deployment tooling to facilitate reuse and extension.

Abstract

We introduce KAPSO, a modular framework for autonomous program synthesis and optimization. Given a natural language goal and an evaluation method, KAPSO iteratively performs ideation, code synthesis and editing, execution, evaluation, and learning to improve a runnable artifact toward measurable objectives. Rather than treating synthesis as the endpoint, KAPSO uses synthesis as an operator within a long-horizon optimization loop, where progress is defined by evaluator outcomes. KAPSO targets long-horizon failures common in coding agents, including lost experimental state, brittle debugging, and weak reuse of domain expertise, by integrating three tightly coupled components. First, a git-native experimentation engine isolates each attempt as a branch, producing reproducible artifacts and preserving provenance across iterations. Second, a knowledge system ingests heterogeneous sources, including repositories, internal playbooks, and curated external resources such as documentation, scientific papers, and web search results, and organizes them into a structured representation that supports retrieval over workflows, implementations, and environment constraints. Third, a cognitive memory layer coordinates retrieval and maintains an episodic store of reusable lessons distilled from experiment traces (run logs, diffs, and evaluator feedback), reducing repeated error modes and accelerating convergence. We evaluated KAPSO on MLE-Bench (Kaggle-style ML competitions) and ALE-Bench (AtCoder heuristic optimization), and report end-to-end performance. Code Available at: https://github.com/Leeroo-AI/kapso

Figures (1)

• Figure 1: The overview of the framework architecture.