Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dedicated Inference Engine and Binary-Weight Neural Networks for Lightweight Instance Segmentation

Tse-Wei Chen, Wei Tao, Dongyue Zhao, Kazuhiro Mima, Tadayuki Ito, Kinya Osa, Masami Kato

TL;DR

The paper tackles embedded dense-prediction for instance segmentation by introducing a dedicated inference engine for binary-weight neural networks with two operation modes that replace MACs with bitwise logic. It presents two lightweight networks (MSCAN-SparseInst BNN and ConvNeXtV2-SparseInst BNN) built from SegNeXt backbones and the SparseInst decoder, achieving higher Person-category accuracy than YOLACT while reducing model size by $77.7\times$ and MACs to $9.8\%$ of YOLACT. A hardware analysis shows significant gate-count reductions (about $52\%$ of a modified XNOR multiplier and $59\%$ of a selector-based multiplier) by merging the correction term into the bias and eliminating weight-only computations. The work demonstrates that bitwise-only inference can deliver practical, scalable performance for dense-prediction on embedded devices, enabling real-time instance segmentation with substantially smaller, more energy-efficient hardware.

Abstract

Reducing computational costs is an important issue for development of embedded systems. Binary-weight Neural Networks (BNNs), in which weights are binarized and activations are quantized, are employed to reduce computational costs of various kinds of applications. In this paper, a design methodology of hardware architecture for inference engines is proposed to handle modern BNNs with two operation modes. Multiply-Accumulate (MAC) operations can be simplified by replacing multiply operations with bitwise operations. The proposed method can effectively reduce the gate count of inference engines by removing a part of computational costs from the hardware system. The architecture of MAC operations can calculate the inference results of BNNs efficiently with only 52% of hardware costs compared with the related works. To show that the inference engine can handle practical applications, two lightweight networks which combine the backbones of SegNeXt and the decoder of SparseInst for instance segmentation are also proposed. The output results of the lightweight networks are computed using only bitwise operations and add operations. The proposed inference engine has lower hardware costs than related works. The experimental results show that the proposed inference engine can handle the proposed instance-segmentation networks and achieves higher accuracy than YOLACT on the "Person" category although the model size is 77.7$\times$ smaller compared with YOLACT.

Dedicated Inference Engine and Binary-Weight Neural Networks for Lightweight Instance Segmentation

TL;DR

The paper tackles embedded dense-prediction for instance segmentation by introducing a dedicated inference engine for binary-weight neural networks with two operation modes that replace MACs with bitwise logic. It presents two lightweight networks (MSCAN-SparseInst BNN and ConvNeXtV2-SparseInst BNN) built from SegNeXt backbones and the SparseInst decoder, achieving higher Person-category accuracy than YOLACT while reducing model size by and MACs to of YOLACT. A hardware analysis shows significant gate-count reductions (about of a modified XNOR multiplier and of a selector-based multiplier) by merging the correction term into the bias and eliminating weight-only computations. The work demonstrates that bitwise-only inference can deliver practical, scalable performance for dense-prediction on embedded devices, enabling real-time instance segmentation with substantially smaller, more energy-efficient hardware.

Abstract

Reducing computational costs is an important issue for development of embedded systems. Binary-weight Neural Networks (BNNs), in which weights are binarized and activations are quantized, are employed to reduce computational costs of various kinds of applications. In this paper, a design methodology of hardware architecture for inference engines is proposed to handle modern BNNs with two operation modes. Multiply-Accumulate (MAC) operations can be simplified by replacing multiply operations with bitwise operations. The proposed method can effectively reduce the gate count of inference engines by removing a part of computational costs from the hardware system. The architecture of MAC operations can calculate the inference results of BNNs efficiently with only 52% of hardware costs compared with the related works. To show that the inference engine can handle practical applications, two lightweight networks which combine the backbones of SegNeXt and the decoder of SparseInst for instance segmentation are also proposed. The output results of the lightweight networks are computed using only bitwise operations and add operations. The proposed inference engine has lower hardware costs than related works. The experimental results show that the proposed inference engine can handle the proposed instance-segmentation networks and achieves higher accuracy than YOLACT on the "Person" category although the model size is 77.7 smaller compared with YOLACT.
Paper Structure (7 sections, 6 equations, 7 figures, 6 tables)

This paper contains 7 sections, 6 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Binary-Weight Neural Networks (BNNs)
  3. Proposed Inference Engine
  4. Experimental Results
  5. BNNs for Instance Segmentation
  6. Analysis of Hardware Architectures
  7. Conclusion

Figures (7)

  • Figure 1: Hardware architecture of the proposed dedicated inference engine.
  • Figure 2: Hardware architecture of the "bitwise operation unit" in the dedicated inference engine.
  • Figure 3: Architecture of the proposed network, MSCAN-SparseInst BNN.
  • Figure 4: Architecture of the building block in each stage of MSCAN-SparseInst BNN.
  • Figure 5: Some examples of instance segmentation results of (a) ConvNeXtV2-SparseInst BNN, (b) MSCAN-SparseInst BNN, and (c) YOLACT (ResNet50) Bolya19.
  • ...and 2 more figures