Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks

TL;DR

Model Disassembling and Assembling (MDA), a paradigm that does not require training to obtain new models, is explored and exhibits diverse potential applications, with comprehensive experiments exploring model decision route analysis, model compression, knowledge distillation, and more.

Abstract

With the rapid development of deep learning, the increasing complexity and scale of parameters make training a new model increasingly resource-intensive. In this paper, we start from the classic convolutional neural network (CNN) and explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we draw inspiration from the information subsystem pathways in the biological visual system and propose Model Disassembling and Assembling (MDA). During model disassembling, we introduce the concept of relative contribution and propose a component locating technique to extract task-aware components from trained CNN classifiers. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task, utilizing the disassembled task-aware components. The entire process is akin to playing with LEGO bricks, enabling arbitrary assembly of new models, and providing a novel perspective for model creation and reuse. Extensive experiments showcase that task-aware components disassembled from CNN classifiers or new models assembled using these components closely match or even surpass the performance of the baseline, demonstrating its promising results for model reuse. Furthermore, MDA exhibits diverse potential applications, with comprehensive experiments exploring model decision route analysis, model compression, knowledge distillation, and more. The code is available at https://github.com/jiaconghu/Model-LEGO.

Figures (12)

• Figure 1: Disassembling process at the $l$-th layer of a CNN model, where the red solid line represents the contribution aggregation process, and the black dashed line represents the contribution allocation process.
• Figure 2: Assembling process at the $l$-th layer of CNN models: (a) and (b) represent two distinct disassembled models, respectively; (c) illustrates the assembled model.
• Figure 3: Accuracy curve (a), FLOPs ratio curve (b), and model parameter size ratio curve (c) for the disassembled model, varying with hyper-parameters in the fully connected layers ($\alpha_f,\beta_f$) and convolutional layers ($\alpha_c,\beta_c$).
• Figure 4: Accuracy curve (a), FLOPs ratio curve (b), and model parameter size ratio curve (c) for the disassembled model as the number of disassembled layers varies. The number of disassembled layers accumulates from deep layers to shallow layers in the model.
• Figure 5: The process of contribution aggregation and allocation at the $l$-th fully connected layer, wherein the red solid line delineates the contribution aggregation process, and the black dashed line signifies the contribution allocation process.