Bugs in the Shadows: Static Detection of Faulty Python Refactorings
Jonhnanthan Oliveira, Rohit Gheyi, Márcio Ribeiro, Alessandro Garcia
TL;DR
The paper addresses the risk of type errors during Python refactoring in dynamically typed code by introducing SafeRefactorPy, a static-analysis pipeline that applies real refactorings and compares type-check results before and after using Pyre. By applying 1,152 Rope-based transformations to a real-world project (TextBlob), the approach uncovered 29 bugs across four refactoring types and identified 18 unique type errors, with some issues also surfacing in IDEs like PyCharm and PyDev. The study demonstrates the practical relevance of statically validating refactorings and provides a reproducible workflow for tool developers to catch semantic and type-related errors early in the transformation process. The findings motivate broader validation of Python refactoring tools and suggest integrating static checks (and potentially alternative analyzers like PyType) into IDEs to improve reliability of automated code transformations.
Abstract
Python is a widely adopted programming language, valued for its simplicity and flexibility. However, its dynamic type system poses significant challenges for automated refactoring - an essential practice in software evolution aimed at improving internal code structure without changing external behavior. Understanding how type errors are introduced during refactoring is crucial, as such errors can compromise software reliability and reduce developer productivity. In this work, we propose a static analysis technique to detect type errors introduced by refactoring implementations for Python. We evaluated our technique on Rope refactoring implementations, applying them to open-source Python projects. Our analysis uncovered 29 bugs across four refactoring types from a total of 1,152 refactoring attempts. Several of these issues were also found in widely used IDEs such as PyCharm and PyDev. All reported bugs were submitted to the respective developers, and some of them were acknowledged and accepted. These results highlight the need to improve the robustness of current Python refactoring tools to ensure the correctness of automated code transformations and support reliable software maintenance.