Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Empirical Study of Policy-as-Code Adoption in Open-Source Software Projects

Patrick Loic Foalem, Foutse Khomh, Leuson Da Silva, Ettore Merlo

TL;DR

This paper tackles the problem of understanding how Policy-as-Code (PaC) is adopted and used in real OSS projects. It employs a large-scale, mixed-methods approach on 399 GitHub repositories using nine PaC tools, augmented by an LLM-assisted taxonomy refined through expert validation to categorize PaC usage into five main categories and fourteen sub-categories. Key findings reveal diverse adoption with a strong emphasis on security and compliance governance, predominance of single-tool usage, and notable co-usage patterns such as between OPA and GateKeeper, along with growing but small uptake in MLOps contexts. These insights offer actionable guidance for practitioners and tool developers on tool selection, interoperability, and governance strategy, and establish a foundation for future empirical work on policy-driven trustworthy software systems.

Abstract

\textbf{Context:} Policy-as-Code (PaC) has become a foundational approach for embedding governance, compliance, and security requirements directly into software systems. While organizations increasingly adopt PaC tools, the software engineering community lacks an empirical understanding of how these tools are used in real-world development practices. \textbf{Objective:} This paper aims to bridge this gap by conducting the first large-scale study of PaC usage in open-source software. Our goal is to characterize how PaC tools are adopted, what purposes they serve, and what governance activities they support across diverse software ecosystems. \textbf{Method:} We analyzed 399 GitHub repositories using nine widely adopted PaC tools. Our mixed-methods approach combines quantitative analysis of tool usage and project characteristics with a qualitative investigation of policy files. We further employ a Large Language Model (LLM)--assisted classification pipeline, refined through expert validation, to derive a taxonomy of PaC usage consisting of 5 categories and 15 sub-categories. \textbf{Results:} Our study reveals substantial diversity in PaC adoption. PaC tools are frequently used in early-stage projects and are heavily oriented toward governance, configuration control, and documentation. We also observe emerging PaC usage in MLOps pipelines and strong co-usage patterns, such as between OPA and Gatekeeper. Our taxonomy highlights recurring governance intents. \textbf{Conclusion:} Our findings offer actionable insights for practitioners and tool developers. They highlight concrete usage patterns, emphasize actual PaC usage, and motivate opportunities for improving tool interoperability. This study lays the empirical foundation for future research on PaC practices and their role in ensuring trustworthy, compliant software systems.

An Empirical Study of Policy-as-Code Adoption in Open-Source Software Projects

TL;DR

This paper tackles the problem of understanding how Policy-as-Code (PaC) is adopted and used in real OSS projects. It employs a large-scale, mixed-methods approach on 399 GitHub repositories using nine PaC tools, augmented by an LLM-assisted taxonomy refined through expert validation to categorize PaC usage into five main categories and fourteen sub-categories. Key findings reveal diverse adoption with a strong emphasis on security and compliance governance, predominance of single-tool usage, and notable co-usage patterns such as between OPA and GateKeeper, along with growing but small uptake in MLOps contexts. These insights offer actionable guidance for practitioners and tool developers on tool selection, interoperability, and governance strategy, and establish a foundation for future empirical work on policy-driven trustworthy software systems.

Abstract

\textbf{Context:} Policy-as-Code (PaC) has become a foundational approach for embedding governance, compliance, and security requirements directly into software systems. While organizations increasingly adopt PaC tools, the software engineering community lacks an empirical understanding of how these tools are used in real-world development practices. \textbf{Objective:} This paper aims to bridge this gap by conducting the first large-scale study of PaC usage in open-source software. Our goal is to characterize how PaC tools are adopted, what purposes they serve, and what governance activities they support across diverse software ecosystems. \textbf{Method:} We analyzed 399 GitHub repositories using nine widely adopted PaC tools. Our mixed-methods approach combines quantitative analysis of tool usage and project characteristics with a qualitative investigation of policy files. We further employ a Large Language Model (LLM)--assisted classification pipeline, refined through expert validation, to derive a taxonomy of PaC usage consisting of 5 categories and 15 sub-categories. \textbf{Results:} Our study reveals substantial diversity in PaC adoption. PaC tools are frequently used in early-stage projects and are heavily oriented toward governance, configuration control, and documentation. We also observe emerging PaC usage in MLOps pipelines and strong co-usage patterns, such as between OPA and Gatekeeper. Our taxonomy highlights recurring governance intents. \textbf{Conclusion:} Our findings offer actionable insights for practitioners and tool developers. They highlight concrete usage patterns, emphasize actual PaC usage, and motivate opportunities for improving tool interoperability. This study lays the empirical foundation for future research on PaC practices and their role in ensuring trustworthy, compliant software systems.
Paper Structure (25 sections, 7 figures, 2 tables)

This paper contains 25 sections, 7 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. STUDY DESIGN
  4. RESULTS
  5. RQ1: What is the popularity of different PaC tools in OSS projects?
  6. Motivation
  7. Approach
  8. Result
  9. RQ2: What kind of projects adopt PaC tools?
  10. Motivation
  11. Approach
  12. Result
  13. RQ3: How is PaC used in practice across different projects?
  14. Motivation
  15. Approach
  16. ...and 10 more sections

Figures (7)

  • Figure 1: Policy-as-Code architecture: policies are authored once and enforced consistently by a centralized engine across services and pipelines.
  • Figure 2: Overview of our research workflow.
  • Figure 3: Distribution of project metrics using PaC tools: Age (months), Size (bytes), Stars (popularity), and Contributors (collaboration) across our study projects.
  • Figure 4: Comparison of Policy as Code (PaC) tool usage
  • Figure 5: A Taxonomy of Policy-as-Code Usage and Adoption Patterns
  • ...and 2 more figures