Quantum software experiments: A reporting and laboratory package structure guidelines

TL;DR

The paper addresses the lack of standardized reporting and packaging for quantum software experiments by extending established software engineering guidelines to quantum contexts and by proposing a comprehensive Quantum Laboratory Package (LP) structure. It presents a twofold contribution: (i) tailored reporting guidelines that cover context, planning, design, execution, analysis, and validity for QSEs, and (ii) a detailed LP structure that consolidates articles, materials, datasets, systems, scripts, and dictionaries to enable rigorous replication. The authors validate the proposals through a multi-case empirical review of 11 QSE works (5 on reporting guidelines, 6 on LPs), identifying gaps and demonstrating potential improvements in replicability and verifiability. The work aims to standardize practices in quantum software research, facilitating rigorous, transparent, and transferable experiments across providers and platforms. Overall, the guidelines and LP structure offer a foundational step toward more reliable and reusable quantum software research in an immature, rapidly evolving field.

Abstract

Background. In the realm of software engineering, there are widely accepted guidelines for reporting and creating laboratory packages. Unfortunately, the landscape differs considerably in the emerging field of quantum computing. To the best of our knowledge, no standardized guidelines exist for describing experiments or outlining the necessary structures for quantum software laboratory packages. Aims. This paper endeavors to enhance the replicability and verifiability of quantum software experiments. Method. This objective is pursued through the proposition of guidelines for reporting and the delineation of a structure for laboratory packages tailored to quantum computing experiments. Specifically, we advocate for an extension and adaption of established guidelines in experimental software engineering, integrating novel elements to address the specific requirements of quantum software engineering. Results. In validating the utility and effectiveness of the proposed guidelines, we conducted a review encompassing 11 works (5 focusing on reporting guidelines and 6 on laboratory packages). In particular, this review highlighted the absence of standardized guidelines and structure of laboratory packages for quantum software experiments. Conclusions. Our assessment revealed gaps in information and opportunities for enhancement within the evaluated papers and laboratory packages. Our proposal contributes to the advancement of quantum software engineering research, taking a fundamental step toward fostering rigorous and reliable scientific research in this emerging paradigm.

Figures (1)

• Figure 1: Example of a quantum circuit