AutoAppendix: Towards one-click Reproduction of Computational Artifacts
Klaus Kraßnitzer
TL;DR
AutoAppendix presents a structured study of SC24 artifact reproducibility on the Chameleon Cloud, identifying practical barriers to repeatability and proposing automation through Trovi templates. It analyzes 18 high-relevance submissions eligible for robust replication, highlighting documentation gaps, hardware dependencies, and variability across environments. The work contributes concrete guidelines for authors, plus three Trovi templates to standardize and simplify artifact creation and evaluation. Together, these efforts aim to lower barriers to reproducibility and promote more reliable, automatable artifact workflows in HPC research.
Abstract
This report summarizes the findings of the AutoAppendix project, conducted during the UCSC OSPO Summer of Reproducibility 2024. The project involved a evaluation of reproducibility artifacts submitted to SC24, focusing on their deployability and robustness on the Chameleon Cloud platform. This technical report aims to inform and support the reproducibility community by sharing observed challenges, patterns, and best practices. Furthermore, we share templates developed for Chameleon Cloud's Jupyter interface that are intended to assist future authors and reviewers in streamlining artifact evaluation workflows.