DeepCode: Open Agentic Coding

TL;DR

The paper tackles the challenge of high-fidelity document-to-repository synthesis, where long, multimodal specifications (e.g., scientific papers) must be translated into executable code under limited context windows. It introduces DeepCode, an information-flow–driven framework that orchestrates four operations—blueprint distillation, stateful memory (CodeMem), retrieval-augmented generation (CodeRAG), and closed-loop verification—to maximize the signal within a constrained context. Empirical results on PaperBench show DeepCode achieving state-of-the-art replication scores, outperforming leading commercial agents and even surpassing PhD-level human experts on key metrics. This work establishes a principled foundation for autonomous scientific reproduction, with potential to accelerate evaluation, reproduction, and discovery by reliably converting complex documents into production-grade implementations.

Abstract

Recent advances in large language models (LLMs) have given rise to powerful coding agents, making it possible for code assistants to evolve into code engineers. However, existing methods still face significant challenges in achieving high-fidelity document-to-codebase synthesis--such as scientific papers to code--primarily due to a fundamental conflict between information overload and the context bottlenecks of LLMs. In this work, we introduce DeepCode, a fully autonomous framework that fundamentally addresses this challenge through principled information-flow management. By treating repository synthesis as a channel optimization problem, DeepCode seamlessly orchestrates four information operations to maximize task-relevant signals under finite context budgets: source compression via blueprint distillation, structured indexing using stateful code memory, conditional knowledge injection via retrieval-augmented generation, and closed-loop error correction. Extensive evaluations on the PaperBench benchmark demonstrate that DeepCode achieves state-of-the-art performance, decisively outperforming leading commercial agents such as Cursor and Claude Code, and crucially, surpassing PhD-level human experts from top institutes on key reproduction metrics. By systematically transforming paper specifications into production-grade implementations comparable to human expert quality, this work establishes new foundations for autonomous scientific reproduction that can accelerate research evaluation and discovery.

Figures (9)

• Figure 1: DeepCode main results.
• Figure 2: From Challenge to Solution of DeepCode. Left: Current AI agents achieve only a 42% paper replication score compared to 72% for human experts, highlighting the limitations of existing agents. Middle: The core challenge stems from information overload conflicting with LLM context limits, causing four key failure modes. Right: DeepCode addresses this through four information operations (Blueprint, CodeMem, CodeRAG, Verification), surpassing human expert performance.
• Figure 3: The overall framework of DeepCode.
• Figure 4: Comparison of DeepCode with four baseline categories: (1) human experts, (2) state-of-the-art commercial code agents, (3) scientific code agents, and (4) LLM-based agents
• Figure 5: DeepCode reproduction results on the 3-paper subset across LLM backbones