Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling Inclusiveness-Related User Feedback in Mobile Applications

Nowshin Nawar Arony, Ze Shi Li, Daniela Damian, Bowen Xu

TL;DR

This study investigates inclusiveness in mobile apps by analyzing end-user feedback from Reddit, Google Play Store, and X for 50 popular apps using a Socio-Technical Grounded Theory framework. It develops a two-layer taxonomy with five high-level categories—Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values—encompassing 14 sub-codes, grounded in 22k+ posts and validated across multiple sources. The authors also evaluate GPT4o-mini for automated identification of inclusiveness-related feedback, finding zero-shot and fine-tuning strategies yield complementary strengths in precision and recall, respectively. The work provides a manually labeled dataset and practical implications for practitioners, highlighting how inclusiveness considerations can translate into requirements and automated tooling to better serve diverse user groups.

Abstract

In an era of rapidly expanding software usage, catering to the diverse needs of users from various backgrounds has become a critical challenge. Inclusiveness, representing a core human value, is frequently overlooked during software development, leading to user dissatisfaction. Users often engage in discourse on online platforms where they indicate their concerns. In this study, we leverage user feedback from three popular online sources Reddit, Google Play Store, and X, for 50 of the most popular apps in the world. Using a Socio-Technical Grounded Theory approach, we analyzed 22,000 posts across the three sources. We organize our empirical results in a taxonomy for inclusiveness comprising 5 major categories: Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values. To explore automated support for identifying inclusiveness-related posts, we experimented with a large language model (GPT4o-mini) and found that it is capable of identifying inclusiveness-related user feedback. We provide implications and recommendations that can help software practitioners to better identify inclusiveness issues to support a wider range of users

Unveiling Inclusiveness-Related User Feedback in Mobile Applications

TL;DR

This study investigates inclusiveness in mobile apps by analyzing end-user feedback from Reddit, Google Play Store, and X for 50 popular apps using a Socio-Technical Grounded Theory framework. It develops a two-layer taxonomy with five high-level categories—Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values—encompassing 14 sub-codes, grounded in 22k+ posts and validated across multiple sources. The authors also evaluate GPT4o-mini for automated identification of inclusiveness-related feedback, finding zero-shot and fine-tuning strategies yield complementary strengths in precision and recall, respectively. The work provides a manually labeled dataset and practical implications for practitioners, highlighting how inclusiveness considerations can translate into requirements and automated tooling to better serve diverse user groups.

Abstract

In an era of rapidly expanding software usage, catering to the diverse needs of users from various backgrounds has become a critical challenge. Inclusiveness, representing a core human value, is frequently overlooked during software development, leading to user dissatisfaction. Users often engage in discourse on online platforms where they indicate their concerns. In this study, we leverage user feedback from three popular online sources Reddit, Google Play Store, and X, for 50 of the most popular apps in the world. Using a Socio-Technical Grounded Theory approach, we analyzed 22,000 posts across the three sources. We organize our empirical results in a taxonomy for inclusiveness comprising 5 major categories: Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values. To explore automated support for identifying inclusiveness-related posts, we experimented with a large language model (GPT4o-mini) and found that it is capable of identifying inclusiveness-related user feedback. We provide implications and recommendations that can help software practitioners to better identify inclusiveness issues to support a wider range of users
Paper Structure (37 sections, 4 figures, 3 tables)

This paper contains 37 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Related Work
  4. Inclusive Software
  5. User Feedback
  6. Methodology
  7. Data Collection
  8. Qualitative Analysis
  9. Automated Analysis
  10. RQ1: A Taxonomy of Inclusiveness
  11. Algorithmic Bias
  12. Terms/Conditions
  13. Recommendation
  14. Technology
  15. Device
  16. ...and 22 more sections

Figures (4)

  • Figure 1: App review for Snapchat from Google Play Store. Underlined text indicates inclusiveness concern.
  • Figure 2: Reddit Post from Discord subreddit. Underlined text indicates inclusiveness concern.
  • Figure 3: Post on X from a user.
  • Figure 4: Taxonomy for inclusiveness-related user feedback from an analysis of Reddit, Google Play and X