How the Future Works at SOUPS: Analyzing Future Work Statements and Their Impact on Usable Security and Privacy Research

TL;DR

The paper investigates how future work statements (FWS) are presented and implemented in usable security and privacy research, focusing on the 2019 SOUPS proceedings and 978 citing works. Through a two-phase systematic analysis, it finds that while FWS are prevalent (129 statements across 26 papers), most are broad or ambiguous and few are implemented or acknowledged in subsequent research. The study reveals limited direct impact of FWS on guiding future work, though thematic influence and methodological reuse are common. It provides concrete recommendations to improve FWS usefulness, including specificity, findability, and explicit acknowledgment, and shares datasets to enable replication and NLP-based analyses.

Abstract

Extending knowledge by identifying and investigating valuable research questions and problems is a core function of research. Research publications often suggest avenues for future work to extend and build upon their results. Considering these suggestions can contribute to developing research ideas that build upon previous work and produce results that tie into existing knowledge. Usable security and privacy researchers commonly add future work statements to their publications. However, our community lacks an in-depth understanding of their prevalence, quality, and impact on future research. Our work aims to address this gap in the research literature. We reviewed all 27 papers from the 2019 SOUPS proceedings and analyzed their future work statements. Additionally, we analyzed 978 publications that cite any paper from SOUPS 2019 proceedings to assess their future work statements' impact. We find that most papers from the SOUPS 2019 proceedings include future work statements. However, they are often unspecific or ambiguous, and not always easy to find. Therefore, the citing publications often matched the future work statements' content thematically, but rarely explicitly acknowledged them, indicating a limited impact. We conclude with recommendations for the usable security and privacy community to improve the utility of future work statements by making them more tangible and actionable, and avenues for future work.

Figures (4)

• Figure 1: Summary of our two-phase systematic literature analysis of 27 papers in the SOUPS 2019 proceedings and their 978 citing research articles.
• Figure 2: Distribution of future work statements in different sections in the 2019 proceedings.
• Figure 3: Summary of future work statement specificity.
• Figure 4: Publication years of that implement a future work statement from 2019 papers.