Neural Network Interoperability Across Platforms

TL;DR

The paper tackles the challenge of migrating neural network code across deep learning frameworks by introducing a pivot-based approach built on the BESSER-NN metamodel, enabling framework-agnostic representation of NN architectures. It implements a three-stage pipeline (AST extraction, transformation to BESSER-NN, and code generation) with template-driven generators for PyTorch and TensorFlow in both directions. Experimental evaluation on five networks demonstrates that migrated models are functionally equivalent to the originals, with MAD values as low as $5.96 \times 10^{-8}$ and accuracy differences up to $0.022$ on benchmark datasets. The work also discusses integration with model-driven system models and outlines future expansion to more frameworks and advanced AI concepts, highlighting practical impact for teams migrating AI software across platforms.

Abstract

The development of smart systems (i.e., systems enhanced with AI components) has thrived thanks to the rapid advancements in neural networks (NNs). A wide range of libraries and frameworks have consequently emerged to support NN design and implementation. The choice depends on factors such as available functionalities, ease of use, documentation and community support. After adopting a given NN framework, organizations might later choose to switch to another if performance declines, requirements evolve, or new features are introduced. Unfortunately, migrating NN implementations across libraries is challenging due to the lack of migration approaches specifically tailored for NNs. This leads to increased time and effort to modernize NNs, as manual updates are necessary to avoid relying on outdated implementations and ensure compatibility with new features. In this paper, we propose an approach to automatically migrate neural network code across deep learning frameworks. Our method makes use of a pivot NN model to create an abstraction of the NN prior to migration. We validate our approach using two popular NN frameworks, namely PyTorch and TensorFlow. We also discuss the challenges of migrating code between the two frameworks and how they were approached in our method. Experimental evaluation on five NNs shows that our approach successfully migrates their code and produces NNs that are functionally equivalent to the originals. Artefacts from our work are available online.

Figures (5)

• Figure 1: The NN metamodel. It illustrates the key NN components and their relationships, including layers, tensorOps, datasets and NN configuration.
• Figure 2: Overview of our migration approach. TensorFlow is used as the source library and PyTorch as the target library. Migration from PyTorch to TensorFlow is also supported. The approach involves three main steps: 1) AST Extraction, 2) Transformation, and 3) Code Generation.
• Figure 3: An illustration of the migration process from TensorFlow to PyTorch, showing the three main steps: 1) AST Extraction, 2) Transformation, and 3) Code Generation. The example focuses on a Dense layer of a neural network defined using Subclassing architecture and shows how the layer name, type, and attributes are mapped to their PyTorch equivalents via the BESSER-NN pivot model.
• Figure 4: Integration of BESSER-NN with the UML language at metamodel level. This class diagram shows only the relevant classes to illustrate the integration.
• Figure 5: Max absolute differences (MAD) for neural networks migrated from TensorFlow to PyTorch. XToY denotes migration from architecture X in the source NN to Y in the target NN. ‘subc’ and ‘seq’ are used to refer to Subclassing and Sequential, respectively.