On the Abolition of the "ICSE Paper" and the Adoption of the "Registered Proposal" and the "Results Report"

TL;DR

The paper tackles the replication crisis and elitism in software engineering by proposing a radical two-tier publication model that replaces the traditional ICSE paper with Registered Proposals and Results Reports. Registered Proposals foreground novel research questions and rigorous evaluation plans, while Results Reports enable independent replication of prior proposals, potentially across multiple teams, with artifacts and protocols central to evaluation. The approach aims to make replication a first-class citizen in the publication pipeline, reduce gatekeeping, and shift incentives toward durable, verifiable findings. If adopted, this model could enhance reproducibility, broaden participation, and better align SE research with real-world practice, though it carries uncertainties around incentives, logistics, and venue organization.

Abstract

To address the 'novelty-vicious cycle' and the 'replicability crisis' of the field (both discussed in the survey) we propose abolishing the "ICSE paper" as we know it and replacing it with a two-tier system that also evolves the existing notion of 'Registered Report'. Authors proposing a new idea, experiment, or analysis would submit a "Registered Proposal" of their idea and the proposed experimental methodology to undergo peer review. The following year, anyone can submit (shorter) "Results Reports" on the realization of the empirical work based on the registered proposals of the previous ICSE (or FSE or ISSTA or ASE etc.). Both works should be first class citizens of the mainstream events. We argue that such a disruptive (heretical?) idea is supported and based on the responses of the community of the Future of Software Engineering pre-survey