Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

app.build: A Production Framework for Scaling Agentic Prompt-to-App Generation with Environment Scaffolding

Evgenii Kniazev, Arseny Kravchenko, Igor Rekun, James Broadhead, Nikita Shamgunov, Pranav Sah, Pratik Nichite, Ivan Yamshchikov

TL;DR

app.build introduces Environment Scaffolding (ES) as an environment-first framework for production-grade LLM-powered app generation. By coupling structured task decomposition with multi-layered, per-step validation and runtime isolation, ES enables model-agnostic repair loops that improve reliability while enabling cost-aware model selection. Across 300 automated experiments and 30 human evaluations on 30 prompts, ES demonstrates that comprehensive validation can achieve high viability (e.g., $V=1$ in 73.3% of cases) and meaningful quality (mean $Q$ around 8.8 for viable apps), with open-weight models offering substantial cost reductions (e.g., $0.61$ per viable app) relative to closed models. Production deployment since 2025 shows real-world adoption (thousands of apps generated), supporting the claim that environment design, not just model capability, drives production reliability. The work provides complete reference implementations, promoting reproducibility and practical adoption in production-oriented agent systems.

Abstract

We present app.build (https://github.com/neondatabase/appdotbuild-agent), an open-source framework that improves LLM-based application generation through systematic validation and structured environments. Our approach combines multi-layered validation pipelines, stack-specific orchestration, and model-agnostic architecture, implemented across three reference stacks. Through evaluation on 30 generation tasks, we demonstrate that comprehensive validation achieves 73.3% viability rate with 30% reaching perfect quality scores, while open-weights models achieve 80.8% of closed-model performance when provided structured environments. The open-source framework has been adopted by the community, with over 3,000 applications generated to date. This work demonstrates that scaling reliable AI agents requires scaling environments, not just models -- providing empirical insights and complete reference implementations for production-oriented agent systems.

app.build: A Production Framework for Scaling Agentic Prompt-to-App Generation with Environment Scaffolding

TL;DR

app.build introduces Environment Scaffolding (ES) as an environment-first framework for production-grade LLM-powered app generation. By coupling structured task decomposition with multi-layered, per-step validation and runtime isolation, ES enables model-agnostic repair loops that improve reliability while enabling cost-aware model selection. Across 300 automated experiments and 30 human evaluations on 30 prompts, ES demonstrates that comprehensive validation can achieve high viability (e.g., in 73.3% of cases) and meaningful quality (mean around 8.8 for viable apps), with open-weight models offering substantial cost reductions (e.g., per viable app) relative to closed models. Production deployment since 2025 shows real-world adoption (thousands of apps generated), supporting the claim that environment design, not just model capability, drives production reliability. The work provides complete reference implementations, promoting reproducibility and practical adoption in production-oriented agent systems.

Abstract

We present app.build (https://github.com/neondatabase/appdotbuild-agent), an open-source framework that improves LLM-based application generation through systematic validation and structured environments. Our approach combines multi-layered validation pipelines, stack-specific orchestration, and model-agnostic architecture, implemented across three reference stacks. Through evaluation on 30 generation tasks, we demonstrate that comprehensive validation achieves 73.3% viability rate with 30% reaching perfect quality scores, while open-weights models achieve 80.8% of closed-model performance when provided structured environments. The open-source framework has been adopted by the community, with over 3,000 applications generated to date. This work demonstrates that scaling reliable AI agents requires scaling environments, not just models -- providing empirical insights and complete reference implementations for production-oriented agent systems.

Paper Structure

This paper contains 29 sections, 2 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. The Production Reliability Gap
  3. Our Approach: Environment Scaffolding
  4. How environment scaffolding works in practice
  5. Why this differs from model-centric approaches
  6. Contributions
  7. Related Work
  8. Industrial Context & System
  9. Problem Formulation
  10. Architecture
  11. Experimental Setup
  12. Evaluation Framework
  13. Prompt Dataset
  14. Experimental Configurations
  15. Assessor Protocol and Scoring
  16. ...and 14 more sections

Figures (4)

  • Figure 1: Environment scaffolding vs. model-centric generation. ES wraps the model with a finite, validated workflow that catches errors early and repairs them before proceeding.
  • Figure 2: app.build architecture expressed through environment scaffolding. The orchestrator plans stages per stack; each sub-task runs in a sandbox, is validated, and only then merged. Continuous Integration/Continuous Deployment (CI/CD) and database provisioning are integrated.
  • Figure 3: GitHub star growth trajectory for appdotbuild/agent repository showing 13x growth over 5 months (May-October 2025), with inflection point in June 2025 coinciding with production deployment launch. The sustained upward trajectory through October 2025 indicates genuine practitioner adoption rather than transient interest. Data from star-history.com.
  • Figure 4: Production usage metrics demonstrating real-world deployment scale. Left: Daily application creation and deployment activity showing peak usage of 220+ apps/day in early August 2025. Right: User growth trajectory over 30 days showing rapid adoption spike coinciding with peak usage period, reaching 160+ active users. Data from production database analytics.