Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization

Nathan Cassee, Neil Ernst, Nicole Novielli, Alexander Serebrenik

TL;DR

This paper investigates whether negativity in self-admitted technical debt (SATD) descriptions biases how developers prioritize debt items. Using a vignette-based, between-subject experiment with 59 participants and four SATD instances (two negative, two neutral), the authors apply Bayesian ordered-logit models to estimate effects on Urgency, Importance, and Effort while controlling for Perception, Manipulated status, and Experience. The results show that 30-50% of developers assign higher priority when negativity is present, with Urgency most affected and a substantial belief-practice gap: two-thirds reject negativity as a proxy for priority. The study discusses implications for practice, noting that negativity should not be used as a proxy despite its unavoidable presence in descriptions, and calls for field studies and alternative prioritization mechanisms.

Abstract

Self-Admitted Technical Debt, or SATD, is a self-admission of technical debt present in a software system. To effectively manage SATD, developers need to estimate its priority and assess the effort required to fix the described technical debt. About a quarter of descriptions of SATD in software systems express some form of negativity or negative emotions when describing technical debt. In this paper, we report on an experiment conducted with 59 respondents to study whether negativity expressed in the description of SATD \textbf{actually} affects the prioritization of SATD. The respondents are a mix of professional developers and students, and in the experiment, we asked participants to prioritize four vignettes: two expressing negativity and two expressing neutral sentiment. To ensure realism, vignettes were based on existing SATD. We find that negativity causes between one-third and half of developers to prioritize SATD, in which negativity is expressed as having more priority. Developers affected by negativity when prioritizing SATD are twice as likely to increase their estimation of urgency and 1.5 times as likely to increase their estimation of importance and effort for SATD compared to the likelihood of decreasing these prioritization scores. Our findings show how developers actively use negativity in SATD to determine how urgently a particular instance of TD should be addressed. However, our study also describes a gap in the actions and belief of developers. Even if 33% to 50% use negativity to prioritize SATD, 67% of developers believe that using negativity as a proxy for priority is unacceptable. Therefore, we would not recommend using negativity as a proxy for priority. However, we also recognize that developers might unavoidably express negativity when describing technical debt.

Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization

TL;DR

This paper investigates whether negativity in self-admitted technical debt (SATD) descriptions biases how developers prioritize debt items. Using a vignette-based, between-subject experiment with 59 participants and four SATD instances (two negative, two neutral), the authors apply Bayesian ordered-logit models to estimate effects on Urgency, Importance, and Effort while controlling for Perception, Manipulated status, and Experience. The results show that 30-50% of developers assign higher priority when negativity is present, with Urgency most affected and a substantial belief-practice gap: two-thirds reject negativity as a proxy for priority. The study discusses implications for practice, noting that negativity should not be used as a proxy despite its unavoidable presence in descriptions, and calls for field studies and alternative prioritization mechanisms.

Abstract

Self-Admitted Technical Debt, or SATD, is a self-admission of technical debt present in a software system. To effectively manage SATD, developers need to estimate its priority and assess the effort required to fix the described technical debt. About a quarter of descriptions of SATD in software systems express some form of negativity or negative emotions when describing technical debt. In this paper, we report on an experiment conducted with 59 respondents to study whether negativity expressed in the description of SATD \textbf{actually} affects the prioritization of SATD. The respondents are a mix of professional developers and students, and in the experiment, we asked participants to prioritize four vignettes: two expressing negativity and two expressing neutral sentiment. To ensure realism, vignettes were based on existing SATD. We find that negativity causes between one-third and half of developers to prioritize SATD, in which negativity is expressed as having more priority. Developers affected by negativity when prioritizing SATD are twice as likely to increase their estimation of urgency and 1.5 times as likely to increase their estimation of importance and effort for SATD compared to the likelihood of decreasing these prioritization scores. Our findings show how developers actively use negativity in SATD to determine how urgently a particular instance of TD should be addressed. However, our study also describes a gap in the actions and belief of developers. Even if 33% to 50% use negativity to prioritize SATD, 67% of developers believe that using negativity as a proxy for priority is unacceptable. Therefore, we would not recommend using negativity as a proxy for priority. However, we also recognize that developers might unavoidably express negativity when describing technical debt.
Paper Structure (26 sections, 1 equation, 7 figures, 2 tables)

This paper contains 26 sections, 1 equation, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Choice of Research Method
  4. Alternatives
  5. Considerations:
  6. Implications:
  7. Experimental Design:
  8. Data Analysis
  9. Prior predictive checks:
  10. Perception and Demographics
  11. Results
  12. Demographics
  13. Negativity's Effect on Prioritization
  14. Perceptions and Self-Reported Behavior
  15. Discussion
  16. ...and 11 more sections

Figures (7)

  • Figure 1: A Directed Acyclic Graph (DAG) illustrating the relation we are analyzing. The color blue denotes the outcome and red the exposure.
  • Figure 2: The theoretical model visualized as a DAG. The color blue denotes the outcome, red is the exposure, and grey denotes confounders.
  • Figure 3: Distributions of the priors for each of the outcome variables.
  • Figure 4: Histograms for respondent age and experience.
  • Figure 5: Distributions of the posterior for each of the outcome variables.
  • ...and 2 more figures