Representation Similarity: A Better Guidance of DNN Layer Sharing for Edge Computing without Training

TL;DR

It is shown that S is extremely highly correlated with merged model's accuracy with Pearson Correlation Coefficient more than other metrics, demonstrating that representation similarity can serve as a strong validation accuracy indicator without ground truth.

Abstract

Edge computing has emerged as an alternative to reduce transmission and processing delay and preserve privacy of the video streams. However, the ever-increasing complexity of Deep Neural Networks (DNNs) used in video-based applications (e.g. object detection) exerts pressure on memory-constrained edge devices. Model merging is proposed to reduce the DNNs' memory footprint by keeping only one copy of merged layers' weights in memory. In existing model merging techniques, (i) only architecturally identical layers can be shared; (ii) requires computationally expensive retraining in the cloud; (iii) assumes the availability of ground truth for retraining. The re-evaluation of a merged model's performance, however, requires a validation dataset with ground truth, typically runs at the cloud. Common metrics to guide the selection of shared layers include the size or computational cost of shared layers or representation size. We propose a new model merging scheme by sharing representations (i.e., outputs of layers) at the edge, guided by representation similarity S. We show that S is extremely highly correlated with merged model's accuracy with Pearson Correlation Coefficient |r| > 0.94 than other metrics, demonstrating that representation similarity can serve as a strong validation accuracy indicator without ground truth. We present our preliminary results of the newly proposed model merging scheme with identified challenges, demonstrating a promising research future direction.

Figures (4)

• Figure 1: Proposed merging scheme: sharing representations. DNNs are denoted in different colors. Layers with solid borders represent the weights are loaded in memory while dashed borders indicate the layer weights are offloaded. F: representation.
• Figure 2: Same-Stage (a) (b) and Cross-Stage (c) representation similarity heatmaps for EfficientNet-B0 and EfficientDet-D0. Stages with higher similarity are visualized in brighter colors while darker ones depict dissimilar representations.
• Figure 3: Stage-Wise absolute value of Pearson Correlation Coefficient $r$ between merged accuracy (Acc.) and representation similarity (S) using different metrics. Results indicate a highly correlated relationship ($|r|$ = 0.94) between Acc. and $S$.
• Figure 4: Pearson Correlation Coefficient $r$ between accuracy (Acc.) and representation similarity (S). Each dot denotes a merged model's Acc. and S after representation sharing. (a): Same-Stage similarity; (b): Cross-Stage similarity.