Accountability Capture: How Record-Keeping to Support AI Transparency and Accountability (Re)shapes Algorithmic Oversight

TL;DR

This paper investigates how record-keeping intended to enable AI transparency and accountability can itself reshape socio-technical systems and oversight regimes. It introduces the concept of accountability capture, grounded in Agre's capture theory and Bovens' accountability model, and substantiates it with a survey of 100 practitioners across sectors. The work analyzes how legal, regulatory, and internal demands drive records, the lifecycle of record-keeping practices, and the downstream surveillance and privacy implications for organisations and individuals. It highlights positive effects such as improved compliance and transparency, as well as risks including employee resistance, evasion, and potential data protection concerns, calling for careful evaluation by policymakers and practitioners.

Abstract

Accountability regimes typically encourage record-keeping to enable the transparency that supports oversight, investigation, contestation, and redress. However, implementing such record-keeping can introduce considerations, risks, and consequences, which so far remain under-explored. This paper examines how record-keeping practices bring algorithmic systems within accountability regimes, providing a basis to observe and understand their effects. For this, we introduce, describe, and elaborate 'accountability capture' -- the re-configuration of socio-technical processes and the associated downstream effects relating to record-keeping for algorithmic accountability. Surveying 100 practitioners, we evidence and characterise record-keeping issues in practice, identifying their alignment with accountability capture. We further document widespread record-keeping practices, tensions between internal and external accountability requirements, and evidence of employee resistance to practices imposed through accountability capture. We discuss these and other effects for surveillance, privacy, and data protection, highlighting considerations for algorithmic accountability communities. In all, we show that implementing record-keeping to support transparency in algorithmic accountability regimes can itself bring wider implications -- an issue requiring greater attention from practitioners, researchers, and policymakers alike.

Figures (3)

• Figure 1: Drivers for organisational record-keeping.
• Figure 1: Reported participant demographics, with question text described in captions.
• Figure 2: Organisational effects of imposed record-keeping.