Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest

Altino Alves, Andre Hora

TL;DR

TestMigrationsInPy addresses the need for automated support in migrating Python tests from unittest to pytest, a common yet time-consuming task in software projects. The authors build a dataset of 923 real-world migrations by automatically detecting migration commits and manually isolating pure unittest-to-pytest changes, with per-migration type annotations (e.g., assertion vs. fixture changes). This is positioned as the first dataset focused on testing-framework migration, hosted publicly to enable ground-truth evaluation and validation of migration tools, including LLM-based approaches. The work enables systematic study and development of automated migration techniques by providing precise diffs, summaries, and a scalable benchmark across 100 popular Python projects.

Abstract

Unittest and pytest are the most popular testing frameworks in Python. Overall, pytest provides some advantages, including simpler assertion, reuse of fixtures, and interoperability. Due to such benefits, multiple projects in the Python ecosystem have migrated from unittest to pytest. To facilitate the migration, pytest can also run unittest tests, thus, the migration can happen gradually over time. However, the migration can be time-consuming and take a long time to conclude. In this context, projects would benefit from automated solutions to support the migration process. In this paper, we propose TestMigrationsInPy, a dataset of test migrations from unittest to pytest. TestMigrationsInPy contains 923 real-world migrations performed by developers. Future research proposing novel solutions to migrate frameworks in Python can rely on TestMigrationsInPy as a ground truth. Moreover, as TestMigrationsInPy includes information about the migration type (e.g., changes in assertions or fixtures), our dataset enables novel solutions to be verified effectively, for instance, from simpler assertion migrations to more complex fixture migrations. TestMigrationsInPy is publicly available at: https://github.com/altinoalvesjunior/TestMigrationsInPy.

TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest

TL;DR

TestMigrationsInPy addresses the need for automated support in migrating Python tests from unittest to pytest, a common yet time-consuming task in software projects. The authors build a dataset of 923 real-world migrations by automatically detecting migration commits and manually isolating pure unittest-to-pytest changes, with per-migration type annotations (e.g., assertion vs. fixture changes). This is positioned as the first dataset focused on testing-framework migration, hosted publicly to enable ground-truth evaluation and validation of migration tools, including LLM-based approaches. The work enables systematic study and development of automated migration techniques by providing precise diffs, summaries, and a scalable benchmark across 100 popular Python projects.

Abstract

Unittest and pytest are the most popular testing frameworks in Python. Overall, pytest provides some advantages, including simpler assertion, reuse of fixtures, and interoperability. Due to such benefits, multiple projects in the Python ecosystem have migrated from unittest to pytest. To facilitate the migration, pytest can also run unittest tests, thus, the migration can happen gradually over time. However, the migration can be time-consuming and take a long time to conclude. In this context, projects would benefit from automated solutions to support the migration process. In this paper, we propose TestMigrationsInPy, a dataset of test migrations from unittest to pytest. TestMigrationsInPy contains 923 real-world migrations performed by developers. Future research proposing novel solutions to migrate frameworks in Python can rely on TestMigrationsInPy as a ground truth. Moreover, as TestMigrationsInPy includes information about the migration type (e.g., changes in assertions or fixtures), our dataset enables novel solutions to be verified effectively, for instance, from simpler assertion migrations to more complex fixture migrations. TestMigrationsInPy is publicly available at: https://github.com/altinoalvesjunior/TestMigrationsInPy.
Paper Structure (14 sections, 5 figures)

This paper contains 14 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Dataset Construction
  3. Detecting Migrations from Unittest to Pytest
  4. Detecting Isolated Migrations
  5. Dataset Description
  6. Dataset Examples
  7. Example 1: Simple Migration of Assertions
  8. Example 2: Complex Migration of Fixtures and Assertions
  9. Dataset Usage: Support the Development of Novel Framework Migration Solutions
  10. Limitations
  11. Related Work
  12. Code Migration
  13. Datasets to Support Testing Research
  14. Conclusion

Figures (5)

  • Figure 1: Migration from unittest to pytest (Termgraph).
  • Figure 2: Example of output.info file (Saleor, commit #1).
  • Figure 3: Example of migration of asserts (Saleor, commit #1, migration #1)
  • Figure 4: Example of output.info file (Dash, commit #1).
  • Figure 5: Example of migration fixtures and assertions (Dash, commit #1, migration #4)