A Comprehensive Study of Deep Learning Model Fixing Approaches
Hanmo You, Zan Wang, Zishuo Dong, Luanqi Mo, Jianjun Zhao, Junjie Chen
TL;DR
DL faults threaten safety and reliability, motivating the need for robust repair strategies. The authors perform a large-scale, uniform evaluation of 16 fixing approaches spanning model-, layer-, and neuron-level granularity, assessing correctness and side properties such as robustness, fairness, and backward compatibility. Key findings show model-level fixes are most effective for correctness but cannot simultaneously optimize all properties, and performance deteriorates with model size, underscoring the need for multi-property mitigation and scalable solutions. The work provides practical guidance for industry adoption, outlines implications for future research, and contributes an open evaluation platform with replication data to advance reproducibility.
Abstract
Deep Learning (DL) has been widely adopted in diverse industrial domains, including autonomous driving, intelligent healthcare, and aided programming. Like traditional software, DL systems are also prone to faults, whose malfunctioning may expose users to significant risks. Consequently, numerous approaches have been proposed to address these issues. In this paper, we conduct a large-scale empirical study on 16 state-of-the-art DL model fixing approaches, spanning model-level, layer-level, and neuron-level categories, to comprehensively evaluate their performance. We assess not only their fixing effectiveness (their primary purpose) but also their impact on other critical properties, such as robustness, fairness, and backward compatibility. To ensure comprehensive and fair evaluation, we employ a diverse set of datasets, model architectures, and application domains within a uniform experimental setup for experimentation. We summarize several key findings with implications for both industry and academia. For example, model-level approaches demonstrate superior fixing effectiveness compared to others. No single approach can achieve the best fixing performance while improving accuracy and maintaining all other properties. Thus, academia should prioritize research on mitigating these side effects. These insights highlight promising directions for future exploration in this field.