Shannon Entropy is better Feature than Category and Sentiment in User Feedback Processing
Andres Rojas Paredes, Brenda Mareco
TL;DR
This study tackles how to prioritize user reviews for app development by ranking them with a weighted feature function. It shows that Shannon Entropy, computed directly from review text, can replace traditional features such as category, sentiment, and length, yielding equal or better $NDCG$ scores than standard feature sets under an exhaustive two-decimal weight search. However, increasing weight precision reveals formidable computational limits, and fairness concerns arise as country bias affects rankings; bias mitigation reduces ranking quality, highlighting a trade-off between accuracy and fairness. Overall, entropy-based features emerge as a promising, computationally lightweight alternative to classifier-dependent attributes, warranting further exploration of entropy-related metrics in requirements engineering feedback processing.
Abstract
App reviews in mobile app stores contain useful information which is used to improve applications and promote software evolution. This information is processed by automatic tools which prioritize reviews. In order to carry out this prioritization, reviews are decomposed into features like category and sentiment. Then, a weighted function assigns a weight to each feature and a review ranking is calculated. Unfortunately, in order to extract category and sentiment from reviews, its is required at least a classifier trained in an annotated corpus. Therefore this task is computational demanding. Thus, in this work, we propose Shannon Entropy as a simple feature which can replace standard features. Our results show that a Shannon Entropy based ranking is better than a standard ranking according to the NDCG metric. This result is promising even if we require fairness by means of algorithmic bias. Finally, we highlight a computational limit which appears in the search of the best ranking.