QiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a Neural-Symbolic Approach

TL;DR

QiMeng-Xpiler tackles the problem of Write Once, Run Anywhere transcompilation for tensor programs across heterogeneous deep learning systems by combining LLM-generated transformation passes with small-scale SMT-based repairs. The approach decomposes transcompilation into neural-guided passes and uses hierarchical auto-tuning (intra-pass and inter-pass via MCTS) to optimize both parameters and pass sequences. Empirical results across four DLS show high translation accuracy (around 95% compilation accuracy, 86.9–100% computation accuracy) and up to 2× speedups over vendor libraries, with substantial productivity gains (up to 96×). The work demonstrates the practical viability of neural-symbolic synthesis for cross-platform tensor programming and highlights remaining challenges in handling highly complex control flows.

Abstract

Heterogeneous deep learning systems (DLS) such as GPUs and ASICs have been widely deployed in industrial data centers, which requires to develop multiple low-level tensor programs for different platforms. An attractive solution to relieve the programming burden is to transcompile the legacy code of one platform to others. However, current transcompilation techniques struggle with either tremendous manual efforts or functional incorrectness, rendering "Write Once, Run Anywhere" of tensor programs an open question. We propose a novel transcompiler, i.e., QiMeng-Xpiler, for automatically translating tensor programs across DLS via both large language models (LLMs) and symbolic program synthesis, i.e., neural-symbolic synthesis. The key insight is leveraging the powerful code generation ability of LLM to make costly search-based symbolic synthesis computationally tractable. Concretely, we propose multiple LLM-assisted compilation passes via pre-defined meta-prompts for program transformation. During each program transformation, efficient symbolic program synthesis is employed to repair incorrect code snippets with a limited scale. To attain high performance, we propose a hierarchical auto-tuning approach to systematically explore both the parameters and sequences of transformation passes. Experiments on 4 DLS with distinct programming interfaces, i.e., Intel DL Boost with VNNI, NVIDIA GPU with CUDA, AMD MI with HIP, and Cambricon MLU with BANG, demonstrate that QiMeng-Xpiler correctly translates different tensor programs at the accuracy of 95% on average, and the performance of translated programs achieves up to 2.0x over vendor-provided manually-optimized libraries. As a result, the programming productivity of DLS is improved by up to 96.0x via transcompiling legacy tensor programs.

Figures (9)

• Figure 1: Comparing QiMeng-Xpiler to existing transcompilation techniques in terms of (a) scalability-accuracy axis and (b) human efforts
• Figure 2: The unsuccessful transcompilation examples of GPT-4: (a) Parallelism-related. (b) Memory-related. (c) Instruction-related.
• Figure 3: The overview of QiMeng-Xpiler, a novel transcompiler for automatic transcompilation of tensor programs across different programming models. The transcompiler consists of two parts: (a) neural-symbolic program synthesis, which utilizes LLM to transform code and repair incorrect transformation through symbolic synthesis with limited scales, and (b) hierarchical performance auto-tuning, which systemically explores both the parameters and sequences of transformation passes.
• Figure 4: An illustrative example of the proposed neural-symbolic program synthesis on a tensorization case.
• Figure 5: The SMT constraints for loop split and cache read.