BiDense: Binarization for Dense Prediction

TL;DR

BiDense, a generalized binary neural network designed for efficient and accurate dense prediction tasks, incorporates two key techniques: the Distribution-adaptive Binarizer (DAB) and the Channel-adaptive Full-precision Bypass (CFB).

Abstract

Dense prediction is a critical task in computer vision. However, previous methods often require extensive computational resources, which hinders their real-world application. In this paper, we propose BiDense, a generalized binary neural network (BNN) designed for efficient and accurate dense prediction tasks. BiDense incorporates two key techniques: the Distribution-adaptive Binarizer (DAB) and the Channel-adaptive Full-precision Bypass (CFB). The DAB adaptively calculates thresholds and scaling factors for binarization, effectively retaining more information within BNNs. Meanwhile, the CFB facilitates full-precision bypassing for binary convolutional layers undergoing various channel size transformations, which enhances the propagation of real-valued signals and minimizes information loss. By leveraging these techniques, BiDense preserves more real-valued information, enabling more accurate and detailed dense predictions in BNNs. Extensive experiments demonstrate that our framework achieves performance levels comparable to full-precision models while significantly reducing memory usage and computational costs.

Figures (9)

• Figure 1: BiDense approaches the performance of full-precision 32-bit networks in dense prediction tasks while significantly reducing memory and computational costs.
• Figure 2: Overview of a BiDense convolutional layer. The BiDense layer integrates two techniques: 1) The Distribution-adaptive Binarizer (DAB) adaptively binarizes activations based on input distributions, allowing it to retain more real-valued information. 2) The Channel-Adaptive Full-precision Bypass (CFB) employs fusion-down and fusion-up to dynamically adjust the channel size of full-precision features, ensuring they align with the varying shapes of binary convolutional outputs while introducing minimal operations, which facilitates the propagation of real-valued and fine-grained information. Bi-Conv represents binary convolution. The red arrow $\downarrow$ indicates the flow of full-precision information, while the blue arrow $\downarrow$ denotes the flow of the binarized signal.
• Figure 3: Average channel-wise information entropy and feature visualization of activations before and after the first binarization in networks on the ADE20K zhou2017scene validation set. The feature visualization is represented by averaging the activations. As indicated by information entropy, the fixed thresholds in BNN and ReActNet can result in suboptimal binarization and information loss, whereas BiDense retains more information by employing adaptive thresholds in the Distribution-adaptive Binarizer (DAB).
• Figure 4: Qualitative Results of semantic segmentation on the ADE20K zhou2017scene validation set. BiDense preserves full-precision semantic information, yielding more fine-grained and accurate segmentation than previous BNNs.
• Figure 5: Qualitative results of monocular depth estimation results on the NYUv2 silberman2012indoor and KITTI geiger2012we depth datasets. BiDense retains real-valued information, resulting in more accurate performance and finer-grained depth predictions compared to previous BNNs.