Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformer Architecture for NetsDB

Subodh Kamble, Kunal Sunil Kasodekar

TL;DR

The paper addresses efficient deployment of transformer encoders by integrating a two-block Transformer Encoder into NetsDB, a relational database-powered model-serving platform. It presents an end-to-end NetsDB implementation, a pseudocode reference, and a cross-framework performance analysis against PyTorch, TensorFlow, Flax, and MxNet, highlighting inference latency and model size. Containerization with Singularity on HPC ensures reproducibility while discussing NetsDB's model deduplication and storage optimization benefits. The study demonstrates the viability and trade-offs of DB-backed model serving for large-scale neural architectures and provides a benchmarking framework for future cross-framework comparisons.

Abstract

Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to use relational databases for deploying large-scale deep learning models and created an open-source implementation called NetsDB for the same. We build upon the previous work of these authors by creating an end-to-end implementation of the Encoder part of the transformer for model serving in NetsDB. Specifically, we construct a two-block encoder that includes Multi-Head Attention and its accompanying self-attention mechanism, Layer-Norm, Dropout, FeedForward Layers, and the necessary residual connections. We load out weights from our model for distributed processing, deployment, and efficient inferencing. To prove the efficacy of our implementation, we conduct a comprehensive performance analysis by comparing it with existing implementations in PyTorch, Tensorflow, Flax, and MxNet across key metrics such as inference time and model size.

Transformer Architecture for NetsDB

TL;DR

The paper addresses efficient deployment of transformer encoders by integrating a two-block Transformer Encoder into NetsDB, a relational database-powered model-serving platform. It presents an end-to-end NetsDB implementation, a pseudocode reference, and a cross-framework performance analysis against PyTorch, TensorFlow, Flax, and MxNet, highlighting inference latency and model size. Containerization with Singularity on HPC ensures reproducibility while discussing NetsDB's model deduplication and storage optimization benefits. The study demonstrates the viability and trade-offs of DB-backed model serving for large-scale neural architectures and provides a benchmarking framework for future cross-framework comparisons.

Abstract

Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to use relational databases for deploying large-scale deep learning models and created an open-source implementation called NetsDB for the same. We build upon the previous work of these authors by creating an end-to-end implementation of the Encoder part of the transformer for model serving in NetsDB. Specifically, we construct a two-block encoder that includes Multi-Head Attention and its accompanying self-attention mechanism, Layer-Norm, Dropout, FeedForward Layers, and the necessary residual connections. We load out weights from our model for distributed processing, deployment, and efficient inferencing. To prove the efficacy of our implementation, we conduct a comprehensive performance analysis by comparing it with existing implementations in PyTorch, Tensorflow, Flax, and MxNet across key metrics such as inference time and model size.
Paper Structure (20 sections, 4 figures, 2 tables)

This paper contains 20 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Why NetsDB?
  3. Background on the Transformer Encoder Architecture
  4. Related Work
  5. Experimental Setup
  6. Overall Issues faced
  7. Singularity
  8. Methodology
  9. Systems for Comparison
  10. General workflow across all the frameworks
  11. PseudoCode for Encoder Block
  12. Implementation of Transformer Block in NetsDB
  13. Other comparative Implementation
  14. PyTorch
  15. TensorFlow
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Encoder Block
  • Figure 2: Self-Attention
  • Figure 3: Multi-Head Attention
  • Figure 4: Flowchart