Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ADELT: Transpilation Between Deep Learning Frameworks

Linyuan Gong, Jiayi Wang, Alvin Cheung

TL;DR

ADELT tackles the challenge of translating deep learning code between frameworks without labeled data. It decouples code skeleton transpilation, handled via few-shot prompts on a large language model, from API keyword mapping, learned through domain-adversarial training on PyBERT contextual embeddings to produce a hierarchical API dictionary. Trained on an unlabeled web corpus, ADELT achieves high pass@1 rates and outperforms state-of-the-art LLM baselines (e.g., GPT-4) while offering faster runtimes due to its dictionary-based mapping component. The approach provides open-source code and demonstrates robust translation across PyTorch–Keras and PyTorch–MXNet pairs, highlighting practical impact for cross-framework DL code porting.

Abstract

We propose the Adversarial DEep Learning Transpiler (ADELT), a novel approach to source-to-source transpilation between deep learning frameworks. ADELT uniquely decouples code skeleton transpilation and API keyword mapping. For code skeleton transpilation, it uses few-shot prompting on large language models (LLMs), while for API keyword mapping, it uses contextual embeddings from a code-specific BERT. These embeddings are trained in a domain-adversarial setup to generate a keyword translation dictionary. ADELT is trained on an unlabeled web-crawled deep learning corpus, without relying on any hand-crafted rules or parallel data. It outperforms state-of-the-art transpilers, improving pass@1 rate by 17.4 pts and 15.0 pts for PyTorch-Keras and PyTorch-MXNet transpilation pairs respectively. We provide open access to our code at https://github.com/gonglinyuan/adelt.

ADELT: Transpilation Between Deep Learning Frameworks

TL;DR

ADELT tackles the challenge of translating deep learning code between frameworks without labeled data. It decouples code skeleton transpilation, handled via few-shot prompts on a large language model, from API keyword mapping, learned through domain-adversarial training on PyBERT contextual embeddings to produce a hierarchical API dictionary. Trained on an unlabeled web corpus, ADELT achieves high pass@1 rates and outperforms state-of-the-art LLM baselines (e.g., GPT-4) while offering faster runtimes due to its dictionary-based mapping component. The approach provides open-source code and demonstrates robust translation across PyTorch–Keras and PyTorch–MXNet pairs, highlighting practical impact for cross-framework DL code porting.

Abstract

We propose the Adversarial DEep Learning Transpiler (ADELT), a novel approach to source-to-source transpilation between deep learning frameworks. ADELT uniquely decouples code skeleton transpilation and API keyword mapping. For code skeleton transpilation, it uses few-shot prompting on large language models (LLMs), while for API keyword mapping, it uses contextual embeddings from a code-specific BERT. These embeddings are trained in a domain-adversarial setup to generate a keyword translation dictionary. ADELT is trained on an unlabeled web-crawled deep learning corpus, without relying on any hand-crafted rules or parallel data. It outperforms state-of-the-art transpilers, improving pass@1 rate by 17.4 pts and 15.0 pts for PyTorch-Keras and PyTorch-MXNet transpilation pairs respectively. We provide open access to our code at https://github.com/gonglinyuan/adelt.
Paper Structure (27 sections, 2 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 27 sections, 2 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Canonicalization & API Keyword Extraction
  4. Canonicalization.
  5. API keyword extraction.
  6. Skeletal Code Transpilation
  7. Domain-Adversarial Training
  8. PyBERT.
  9. Generator and discriminator.
  10. Output embeddings.
  11. Adversarial training.
  12. Hierarchical API Dictionary Generation
  13. Greedy match
  14. Structure of API keywords.
  15. Hierarchical API dictionary generation
  16. ...and 12 more sections

Figures (2)

  • Figure 1: An example of ADELT's pipeline: an import statement in the code skeleton is transpiled from PyTorch to Keras by a language model via few-shot prompting; a linear fully-connected layer is transpiled by removing the argument in_features and renaming other API keywords according to the learned dictionary. The number (1 to 5) near each arrow label corresponds to the step number in \ref{['sec:method']}.
  • Figure 2: ADELT's domain-adversarial training with contextual embeddings from a PyBERT. The generator and the PyBERT are shared between different DL frameworks. We do not fine-tune the PyBERT during adversarial training.