Table of Contents
Fetching ...

Chaos Engineering in the Wild: Findings from GitHub

Joshua Owotogbe, Indika Kumara, Dario Di Nucci, Damian Andrew Tamburri, Willem-Jan van den Heuvel

TL;DR

This study investigates the adoption of Chaos Engineering (CE) tools in open-source GitHub projects by mining 971 repositories that use 10 popular CE tools. It shows rapid growth in CE tool usage since 2016, with a peak in tool releases around 2018 and sustained activity for tools like Toxiproxy and Chaos Mesh, which dominate adoption and development activity. Repositories are largely development-focused and industry-owned, while fault injections are most commonly network- and instance-termination-related, with relatively few using application-level faults. The work provides practical guidance for practitioners on tool selection and highlights opportunities for researchers to explore underrepresented fault types and domains, especially in education and academia. It also points to a need for broader platform coverage and normalization against overall repository growth to better quantify adoption trends.

Abstract

Chaos engineering aims to improve the resilience of software systems by intentionally injecting faults to identify and address system weaknesses that cause outages in production environments. Although many tools for chaos engineering exist, their practical adoption is not yet explored. This study examines 971 GitHub repositories that incorporate 10 popular chaos engineering tools to identify patterns and trends in their use. The analysis reveals that Toxiproxy and Chaos Mesh are the most frequently used, showing consistent growth since 2016 and reflecting increasing adoption in cloud-native development. The release of new chaos engineering tools peaked in 2018, followed by a shift toward refinement and integration, with Chaos Mesh and LitmusChaos leading in ongoing development activity. Software development is the most frequent application (58.0%), followed by unclassified purposes (16.2%), teaching (10.3%), learning (9.9%), and research (5.7%). Development-focused repositories tend to have higher activity, particularly for Toxiproxy and Chaos Mesh, highlighting their industrial relevance. Fault injection scenarios mainly address network disruptions (40.9%) and instance termination (32.7%), while application-level faults remain underrepresented (3.0%), highlighting for future exploration.

Chaos Engineering in the Wild: Findings from GitHub

TL;DR

This study investigates the adoption of Chaos Engineering (CE) tools in open-source GitHub projects by mining 971 repositories that use 10 popular CE tools. It shows rapid growth in CE tool usage since 2016, with a peak in tool releases around 2018 and sustained activity for tools like Toxiproxy and Chaos Mesh, which dominate adoption and development activity. Repositories are largely development-focused and industry-owned, while fault injections are most commonly network- and instance-termination-related, with relatively few using application-level faults. The work provides practical guidance for practitioners on tool selection and highlights opportunities for researchers to explore underrepresented fault types and domains, especially in education and academia. It also points to a need for broader platform coverage and normalization against overall repository growth to better quantify adoption trends.

Abstract

Chaos engineering aims to improve the resilience of software systems by intentionally injecting faults to identify and address system weaknesses that cause outages in production environments. Although many tools for chaos engineering exist, their practical adoption is not yet explored. This study examines 971 GitHub repositories that incorporate 10 popular chaos engineering tools to identify patterns and trends in their use. The analysis reveals that Toxiproxy and Chaos Mesh are the most frequently used, showing consistent growth since 2016 and reflecting increasing adoption in cloud-native development. The release of new chaos engineering tools peaked in 2018, followed by a shift toward refinement and integration, with Chaos Mesh and LitmusChaos leading in ongoing development activity. Software development is the most frequent application (58.0%), followed by unclassified purposes (16.2%), teaching (10.3%), learning (9.9%), and research (5.7%). Development-focused repositories tend to have higher activity, particularly for Toxiproxy and Chaos Mesh, highlighting their industrial relevance. Fault injection scenarios mainly address network disruptions (40.9%) and instance termination (32.7%), while application-level faults remain underrepresented (3.0%), highlighting for future exploration.
Paper Structure (25 sections, 10 figures, 4 tables)

This paper contains 25 sections, 10 figures, 4 tables.

Figures (10)

  • Figure 1: Data collection and analysis process based on sanchez2022mutation and ustunboyacioglu2024data.
  • Figure 2: Yearly repository creation trends for the top 10 chaos engineering tools
  • Figure 3: Number of CE tools released per year
  • Figure 4: Classification of repositories by type
  • Figure 5: Repository type distribution for each tool
  • ...and 5 more figures