The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment

TL;DR

This study introduces algorithm abandonment as an organizational decision to cease creating or using a harmful algorithmic system, and analyzes 40 real-world campaigns to uncover the dynamics leading to abandonment. It identifies a common, yet non-linear, six-phase sequence—the 6 D's: Discovery, Diagnosis, Dissemination, Dialogue, Decision, and Death—through which critiques escalate and potentially drive decommissioning. The authors also catalog seven socio-technical factors that influence whether abandonment occurs and how quickly, ranging from transparency and data access to the regulatory environment and the algorithm's embedding in other systems. Their findings highlight power imbalances and information gaps that impede accountability, while showcasing how collective action, media coverage, and policy interventions can shift outcomes toward abandonment. The work provides a foundation for technologists, regulators, and advocates to design, monitor, and intervene in ways that increase the feasibility and fairness of abandoning harmful algorithms.

Abstract

As more algorithmic systems have come under scrutiny for their potential to inflict societal harms, an increasing number of organizations that hold power over harmful algorithms have chosen (or were required under the law) to abandon them. While social movements and calls to abandon harmful algorithms have emerged across application domains, little academic attention has been paid to studying abandonment as a means to mitigate algorithmic harms. In this paper, we take a first step towards conceptualizing "algorithm abandonment" as an organization's decision to stop designing, developing, or using an algorithmic system due to its (potential) harms. We conduct a thematic analysis of real-world cases of algorithm abandonment to characterize the dynamics leading to this outcome. Our analysis of 40 cases reveals that campaigns to abandon an algorithm follow a common process of six iterative phases: discovery, diagnosis, dissemination, dialogue, decision, and death, which we term the "6 D's of abandonment". In addition, we highlight key factors that facilitate (or prohibit) abandonment, which include characteristics of both the technical and social systems that the algorithm is embedded within. We discuss implications for several stakeholders, including proprietors and technologists who have the power to influence an algorithm's (dis)continued use, FAccT researchers, and policymakers.

Figures (1)

• Figure 1: Event timelines for three example algorithms hrw2023automatedcapp2020primerhoover2023eating. We visualize the time elapsed to each event since each algorithm's release date (x-axis), and label important events by tagging them with one (or multiple) phase(s) from the 6 D's of abandonment (see legend, bottom right). We provide detailed descriptions of each event in Appendix \ref{['apdx:timeline-descriptions']}. Two of the pictured algorithms (the city of Pittsburgh's predictive policing pilot and the National Eating Disorder Association's "Tessa" chatbot) were abandoned, and had a "Death" phase. Takaful, a cash transfer allocation algorithm deployed in Jordan, was not abandoned and is still in use at the time of writing (denoted by the red arrow into the present). These timelines illustrate how the time elapsed between events (e.g., for the algorithm to be discovered, or be abandoned after discovery) varied considerably between the three cases shown.