Testing in the Evolving World of DL Systems:Insights from Python GitHub Projects

TL;DR

This research investigates testing practices within DL projects in GitHub, and quantifies the adoption of testing methodologies, focusing on aspects like test automation, the types of tests, test suite growth rate, and evolution of testing practices across different project versions.

Abstract

In the ever-evolving field of Deep Learning (DL), ensuring project quality and reliability remains a crucial challenge. This research investigates testing practices within DL projects in GitHub. It quantifies the adoption of testing methodologies, focusing on aspects like test automation, the types of tests (e.g., unit, integration, and system), test suite growth rate, and evolution of testing practices across different project versions. We analyze a subset of 300 carefully selected repositories based on quantitative and qualitative criteria. This study reports insights on the prevalence of testing practices in DL projects within the open-source community.

Figures (3)

• Figure 1: TensorFlow Test Suite Growth.
• Figure 2: Keras Test Suite Growth.
• Figure 3: PyTorch Test Suite Growth.