Efficient Deep Learning Board: Training Feedback Is Not All You Need

TL;DR

EfficientDL is an innovative deep learning board designed for automatic performance prediction and component recommendation that can quickly and precisely recommend twenty-seven system components and predict the performance of DL models without requiring any training feedback.

Abstract

Current automatic deep learning (i.e., AutoDL) frameworks rely on training feedback from actual runs, which often hinder their ability to provide quick and clear performance predictions for selecting suitable DL systems. To address this issue, we propose EfficientDL, an innovative deep learning board designed for automatic performance prediction and component recommendation. EfficientDL can quickly and precisely recommend twenty-seven system components and predict the performance of DL models without requiring any training feedback. The magic of no training feedback comes from our proposed comprehensive, multi-dimensional, fine-grained system component dataset, which enables us to develop a static performance prediction model and comprehensive optimized component recommendation algorithm (i.e., α\b{eta}-BO search), removing the dependency on actually running parameterized models during the traditional optimization search process. The simplicity and power of EfficientDL stem from its compatibility with most DL models. For example, EfficientDL operates seamlessly with mainstream models such as ResNet50, MobileNetV3, EfficientNet-B0, MaxViT-T, Swin-B, and DaViT-T, bringing competitive performance improvements. Besides, experimental results on the CIFAR-10 dataset reveal that EfficientDL outperforms existing AutoML tools in both accuracy and efficiency (approximately 20 times faster along with 1.31% Top-1 accuracy improvement than the cutting-edge methods). Source code, pretrained models, and datasets are available at https://github.com/OpenSELab/EfficientDL.

Figures (6)

• Figure 1: The comparison between EfficientDL and other AutoDL frameworks on the CIFAR-10 dataset. Compared to other AutoDL frameworks that take at least 10 hours to discover an optimal model with 8.2M parameters and achieve 90% Top-1 accuracy on CIFAR-10, our EfficientDL identifies an optimal model with 6.5M parameters in just 0.5 hours, achieving a higher performance of 91.31%.
• Figure 2: Top-1 accuracy of DL models with the same dataset run on different hardware.
• Figure 3: Distribution of these datasets utilized for training deep learning models.
• Figure 4: Hierarchical tree of model architecture in our ImageClassEval.
• Figure 5: Comparison of AutoML frameworks. Existing AutoDL frameworks rely on training feedback from actual runs on the hardware. In contrast, our EfficientDL can quickly and precisely recommend twenty-seven system components and predict the performance of DL models without requiring any training feedback, due to our multi-dimensional system component dataset.