MixConv: Mixed Depthwise Convolutional Kernels

TL;DR

This work reveals that single-kernel depthwise convolutions limit model capacity and efficiency. It introduces MixConv, a drop-in depthwise operation that blends multiple kernel sizes across channel groups, enabling multi-scale pattern capture without changing network topology. Coupled with neural architecture search, MixNet architectures achieve state-of-the-art mobile performance on ImageNet and COCO, and demonstrate strong transfer learning results. The findings suggest substantial practical benefits for lightweight vision systems by combining kernel-size diversity with NAS-driven optimization.

Abstract

Depthwise convolution is becoming increasingly popular in modern efficient ConvNets, but its kernel size is often overlooked. In this paper, we systematically study the impact of different kernel sizes, and observe that combining the benefits of multiple kernel sizes can lead to better accuracy and efficiency. Based on this observation, we propose a new mixed depthwise convolution (MixConv), which naturally mixes up multiple kernel sizes in a single convolution. As a simple drop-in replacement of vanilla depthwise convolution, our MixConv improves the accuracy and efficiency for existing MobileNets on both ImageNet classification and COCO object detection. To demonstrate the effectiveness of MixConv, we integrate it into AutoML search space and develop a new family of models, named as MixNets, which outperform previous mobile models including MobileNetV2 [20] (ImageNet top-1 accuracy +4.2%), ShuffleNetV2 [16] (+3.5%), MnasNet [26] (+1.3%), ProxylessNAS [2] (+2.2%), and FBNet [27] (+2.0%). In particular, our MixNet-L achieves a new state-of-the-art 78.9% ImageNet top-1 accuracy under typical mobile settings (<600M FLOPS). Code is at https://github.com/ tensorflow/tpu/tree/master/models/official/mnasnet/mixnet

Figures (9)

• Figure 1: Accuracy vs kernel sizes -- Each point represents a model variant of MobileNet V1mobilenetv117 and V2 mobilenetv218, where model size is represented by point size. Larger kernels lead to more parameters, but the accuracy actually drops down when kernel size is larger than 9x9.
• Figure 2: Mixed depthwise convolution (MixConv) -- Unlike vanilla depthwise convolution that applies a single kernel to all channels, MixConv partitions channels into groups and apply different kernel size to each group.
• Figure 3: A demo of TensorFlow MixConv.
• Figure 4: MixConv performance on ImageNet -- Each point denotes a model with kernel size from 3x3 to 13x13, same as Figure \ref{['fig:limits']}. MixConv is smaller, faster, and achieves higher accuracy than vanilla depthwise convolutions.
• Figure 5: Per-layer impact of kernel size -- s2 denotes stride 2, while others have stride 1.