Methods to Assess the UK Government's Current Role as a Data Provider for AI
Neil Majithia, Elena Simperl
TL;DR
This paper addresses how to quantify the UK government's role as a data provider for AI by introducing two complementary methods: an ablation study using unlearning to remove government data from LLM training and an information leakage study to test recall of data.gov.uk datasets. The ablation method demonstrates that government websites are a valuable knowledge source for LLMs on welfare-related queries, while the information leakage study finds that data.gov.uk is not currently integrated into the training corpora of the tested models. Together, these methods offer a reproducible framework for examining data provenance in AI training, with concrete implications for policy and governance of public data. The work also outlines practical limitations and suggests how the framework could generalize to other datasets and domains, including non-text modalities and copyright considerations.
Abstract
Governments typically collect and steward a vast amount of high-quality data on their citizens and institutions, and the UK government is exploring how it can better publish and provision this data to the benefit of the AI landscape. However, the compositions of generative AI training corpora remain closely guarded secrets, making the planning of data sharing initiatives difficult. To address this, we devise two methods to assess UK government data usage for the training of Large Language Models (LLMs) and 'peek behind the curtain' in order to observe the UK government's current contributions as a data provider for AI. The first method, an ablation study that utilises LLM 'unlearning', seeks to examine the importance of the information held on UK government websites for LLMs and their performance in citizen query tasks. The second method, an information leakage study, seeks to ascertain whether LLMs are aware of the information held in the datasets published on the UK government's open data initiative data$.$gov$.$uk. Our findings indicate that UK government websites are important data sources for AI (heterogenously across subject matters) while data$.$gov$.$uk is not. This paper serves as a technical report, explaining in-depth the designs, mechanics, and limitations of the above experiments. It is accompanied by a complementary non-technical report on the ODI website in which we summarise the experiments and key findings, interpret them, and build a set of actionable recommendations for the UK government to take forward as it seeks to design AI policy. While we focus on UK open government data, we believe that the methods introduced in this paper present a reproducible approach to tackle the opaqueness of AI training corpora and provide organisations a framework to evaluate and maximize their contributions to AI development.