Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SPELL: Synthesis of Programmatic Edits using LLMs

Daniel Ramos, Catarina Gamboa, Inês Lynce, Vasco Manquinho, Ruben Martins, Claire Le Goues

TL;DR

Spell tackles the costly problem of API library migration by extracting actionable migration knowledge from LLMs and grounding it in a reusable PolyglotPiranha DSL. It uses a two-phase pipeline—data distillation to produce validated migration examples, and agentic synthesis to convert those examples into executable transformation scripts—without requiring historical migration corpora. In experiments on Python libraries, Spell generates rich migration exemplars and achieves 61.6% one-shot script-correctness across nine tasks, outperforming Melt. The approach yields maintainable, testable migration assets that can be applied at scale in real-world repositories, bridging LLM capabilities with industrial-grade code transformation tools.

Abstract

Library migration is a common but error-prone task in software development. Developers may need to replace one library with another due to reasons like changing requirements or licensing changes. Migration typically entails updating and rewriting source code manually. While automated migration tools exist, most rely on mining examples from real-world projects that have already undergone similar migrations. However, these data are scarce, and collecting them for arbitrary pairs of libraries is difficult. Moreover, these migration tools often miss out on leveraging modern code transformation infrastructure. In this paper, we present a new approach to automated API migration that sidesteps the limitations described above. Instead of relying on existing migration data or using LLMs directly for transformation, we use LLMs to extract migration examples. Next, we use an Agent to generalize those examples to reusable transformation scripts in PolyglotPiranha, a modern code transformation tool. Our method distills latent migration knowledge from LLMs into structured, testable, and repeatable migration logic, without requiring preexisting corpora or manual engineering effort. Experimental results across Python libraries show that our system can generate diverse migration examples and synthesize transformation scripts that generalize to real-world codebases.

SPELL: Synthesis of Programmatic Edits using LLMs

TL;DR

Spell tackles the costly problem of API library migration by extracting actionable migration knowledge from LLMs and grounding it in a reusable PolyglotPiranha DSL. It uses a two-phase pipeline—data distillation to produce validated migration examples, and agentic synthesis to convert those examples into executable transformation scripts—without requiring historical migration corpora. In experiments on Python libraries, Spell generates rich migration exemplars and achieves 61.6% one-shot script-correctness across nine tasks, outperforming Melt. The approach yields maintainable, testable migration assets that can be applied at scale in real-world repositories, bridging LLM capabilities with industrial-grade code transformation tools.

Abstract

Library migration is a common but error-prone task in software development. Developers may need to replace one library with another due to reasons like changing requirements or licensing changes. Migration typically entails updating and rewriting source code manually. While automated migration tools exist, most rely on mining examples from real-world projects that have already undergone similar migrations. However, these data are scarce, and collecting them for arbitrary pairs of libraries is difficult. Moreover, these migration tools often miss out on leveraging modern code transformation infrastructure. In this paper, we present a new approach to automated API migration that sidesteps the limitations described above. Instead of relying on existing migration data or using LLMs directly for transformation, we use LLMs to extract migration examples. Next, we use an Agent to generalize those examples to reusable transformation scripts in PolyglotPiranha, a modern code transformation tool. Our method distills latent migration knowledge from LLMs into structured, testable, and repeatable migration logic, without requiring preexisting corpora or manual engineering effort. Experimental results across Python libraries show that our system can generate diverse migration examples and synthesize transformation scripts that generalize to real-world codebases.
Paper Structure (22 sections, 1 equation, 5 figures, 3 tables)

This paper contains 22 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Illustrative Example
  3. Migration Data Distillation
  4. Model Sampling
  5. Use Case Generation
  6. Implementation Generation
  7. Test Generation
  8. Migration
  9. Validation
  10. Migration Script Synthesis
  11. PolyglotPiranha
  12. Atomic Rule Inference
  13. Script synthesis
  14. Evaluation
  15. Experimental Setup
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Migration from cryptography.fernet to pycryptodome: function-level encryption logic before and after migration, with the corresponding PolyglotPiranha transformation script extracted from this example.
  • Figure 2: Overview of Spell's pipeline. Given a pair of source and target APIs, Phase 1 distills data from LLMs in the form of use cases. These use cases guide the generation of examples that contain an implementation in the source API, a migration to the target API, and tests that validate both implementations. Phase 2 synthesizes the migration script using an initial set of statically inferred rules and an agentic loop to update them. The final output is a project-agnostic migration script that can be automatically applied to migrate from a source to a target API.
  • Figure 3: Pipeline for generating migration examples using LLMs. Given a source–target library pair, we prompt the LLM to propose abstract use cases leveraging latent knowledge of both libraries (Step 1). For each use case, we generate multiple source library implementations (Step 2), and test suites (Step 3), and migrate them to the target library (Step 4). Validation entails implementation validation (a), migration validation (b), and quality filtering (c). The output is a collection of validated migration triples (implementation, test, migration).
  • Figure 4: Agentic workflow of Spell's approach to migration script synthesis. Our pipeline leverages an atomic rule inference algorithm for creating an initial set of rules, which is then fed to an LLM agent for PolyglotPiranha script generation. The agent iteratively applies changes to the script and gets feedback from the tests and PolyglotPiranha's runtime to refine the rules.
  • Figure 5: A program generated by Melt (top) and Spell (bottom) from a migration example for requests → httpx. Melt's rules contain generic placeholder names and overabstracts; Spell's script includes scoped application and semantically meaningful names.